Where’d Our Ammo Go?
So we went to Dr. Boom to figure out, for our gear, which ranged weapon we should be using. The contestants are:
And we’ve run into a snag. But first off, here’s the setup.
We went to Dr. Boom with 200 bullets in our ammo pouch. Nothing in our bags, just 200 shots. We equipped Tuskbreaker, used Aspect of the Hawk, but used neither Rapid Fire nor trinkets. We popped Fel Mana potions when necessary to maintain our shot rotation.
For that rotation, we used a variation of the popular 1:1 Auto/Steady macro:
/castsequence reset=2/target !Auto Shot, Steady Shot
/script UIErrorsFrame:Clear();
We recorded our combat log, then parsed it to WoWWebStats. Here’s a link to that report.
The report says we did 746DPS. OK, that’s great. Small problem, though; the report says we only shot 177 bullets.
Wha?
We logged out, trashed our combat log, relogged WoW. Went back to the bank, grabbed another 200 bullets, equipped the BBL, recorded our combat log, and repeated the experiment. Here’s the WWS link for that.
755DPS for this gun, how nice it is. 2% more crits, both Auto and Steady shots. That translates to more Kill Commands and more Focus via Go For the Throat. On the surface, it sure looks like we need to put Tuskbreaker in the bank and keep the BBL for fun and profit.
But we only shot 140 bullets, according to WWS. /boggle
Log out, trash log, re-log, bank, 200 bullets, Wolfslayer, start combat log. WWS link. 711DPS and only 151 shots fired.
Where is our ammo going?
For kicks, we did the entire process over again, this time using Tuskbreaker and manually Auto/Steadying. Cr@p on a stick, but we killed Dr. Boom with 19 bullets left. That doesn’t ruin our data, but it affects it.
Regardless, here’s the WWS link. We clipped the bejeezus out of our Auto Shots, sheesh. 727DPS and only 139 shots fired. Same crit-rate as the other Tuskbreaker run, thankfully. Had we done a better job with our shot rotation, the DPS would’ve been more similar, as well.
But on every experiment, WWS “lost” 13% to 30% of our shots. This we cannot explain and we have a hard time reconciling any “this gun has more DPS than this gun” statements until the missing shots are identified.
Any ideas?
ekister on 08 May 2008 at 12:34 pm #
Only one simple answer
You need to spend 150 badges and get the badge X-bow
LOL
I have always wondered if that happened when you clip a shot that the bullet you clipped goes away without doing any damage.
Interesting that your efforts have exposed that, makes me want to go check out mine on Dr. Boom as well.
Thanks for taking the time.
Bomb on 08 May 2008 at 12:45 pm #
Veerry Strange.
At first I thought either missed, blocked, or dodged. But as I am sure you saw there are none of the above shown in the log either.
I want an ammo refund from Bliz if its just disappearing without reason!
Vronak on 08 May 2008 at 1:03 pm #
Missing bullets? Time for CSI:BRK…
Rob on 08 May 2008 at 1:04 pm #
Idk anything about this issue, but for me 200 ammo isn’t enough to get a solid conclusion. This is my testing methodology (which i am sure has flaws too). It’s just different, probably not necessarily better than the BRK method.
Setup recount. Clear it out. It’s identical to WWS now as far as damage recording, or so close that it makes no difference. Shoot until OOM. Record the damage and dps. Extract shot time from that. I am not sure if it records crits, i bet it does. But i don’t look at that. Most important number is total damage when OOM, and dps, (okay two numbers).
Shooting until OOM shows how long you can go in boss fight without drinking (and feel free to buff yourself with mana oil etc), and shows how long you can stay in a fight if you are planting and wailing away. Further the much larger expenditure of ammo means the sample size is larger and you can figure out how the crit rate from that.
Chris on 08 May 2008 at 1:17 pm #
Recount has definitely gotten better, in fact it now shows when the SSO Pendant of Might procs giving you the power of “The Light” which not even the combat log shows correctly…nice to be able to track the little things going on that effect your numbers…
Drotara on 08 May 2008 at 1:21 pm #
It is an interesting problem. But I have to agree with Rob regarding your test scenarios. 200 bullets is kind of a small sample to be basing conclusions on although I use WWS for my Boom tests as well. I like being able to comb through the log.
Regardless of the bullet situation though, I don’t think you are doing the Tuskbreaker justice if you are using it for a 1:1 rotation. It is slow enough where you should be shooting a 3:2 rotation. You are obviously going to get more benefit out of a faster gun if you are only firing a single Steady Shot.
Hope you find your missing bullets.
Deadrabit on 08 May 2008 at 1:24 pm #
I say that you must break down the facts to determin if it was your charting software failure or game failure.
you will have to pour over the combat log and parse it yourself manualy or write a quick prog to parse it for you.
this assumes that the combat log does record every single shot taken.
if all 200 are in the combat log then you know it’s a parsing error.
if not all 200 then you can suspect that combat log is not getting all of it or bug in ammo usage in game and at that point you can drop it into blizzards lap and let them look into it.
Mauryl on 08 May 2008 at 1:26 pm #
If WWS is missing the shots, I’d just use Damage Meter addon and figure out the dps for each gun manually. Equip 200 shot and FD after the last shot as been fired. Look at total damage output, shots fired, combat time and do the math on paper. It’s more work but you can be better assured of accuracy.
On another note now, Damage Meters differentiates between Auto/Steady/Arcane or whatever damage is being applied so you’d be able to see how the different shots compare with each gun.
Vronak on 08 May 2008 at 1:32 pm #
I think the missing bullets are a far more important issue than BRK’s testing methodology. That ammo’s not cheap! There may be an ammo thief about!
I think we should start with a list of suspects. Does TJ have an alibi? (She was talking about getting herself a present…) Ratshag’s priest just made 70 and may need some gold - motive there! And how well do you REALLY know Ego, who just recently arrived on the scene???
And I hesitate to even bring this up, but… was Hobbes with you at the time of the crime? In fact, was he with you EVERY time ammo turned up missing??
Who else should be added to the list?
Anonymous on 08 May 2008 at 1:32 pm #
Im getting similar results analysing WWS logs for threat (I’m a tank), it seems I have lots of shield slams/revenges/devastates missing
guntress on 08 May 2008 at 1:33 pm #
Why not try with 10 bullets and read the chat log manually to figure out what happened anecdotally?
Stoop on 08 May 2008 at 1:33 pm #
They’re with those socks.
Cauthon on 08 May 2008 at 1:34 pm #
You’re on a Mac, so break out the command line:
grep ‘Bigredkitty.*Boom’ /path/to/WoWCombatLog.txt | egrep ‘(Auto|Steady) Shot’ | wc -l
Swap the log file path with wherever you saved your logs; that should tell you how many shots the raw log registered.
Like others say, though, 200 is far, far too few shots for an accurate sample. One or two lucky or unlucky crits will swing your margin of error significantly beyond the inherit dps of the weapons in question.
Mauryl on 08 May 2008 at 1:44 pm #
Think I found the problem.
When you go into the detailed page that breaks down the info for both Auto and Steady shot the number of hits just doesn’t add up. If you mouse over the number of hits for Auto Shot with the Tuskbreaker it shows (Auto Shot (Physical):Nb of landed hits (non-crit)). I think those numbers are just taking into account the normal shots that have hit and doesn’t bother inputting the shots that crit in that info. I may be wrong but that seems to me to be the issue.
Rohan on 08 May 2008 at 1:51 pm #
The only pattern I see is that the slower the weapon, the fewer shots you are missing. You may have to search the raw combat logs manually.
I’d try firing 200 Autoshots, with no specials, and see if they all show up properly.
Mauryl on 08 May 2008 at 1:53 pm #
Scratch my last. Took a closer look and did the math. Doesn’t come out to 200 shots.
Makers on 08 May 2008 at 2:39 pm #
maybe a dingo ate your bullets!
Capn John on 08 May 2008 at 2:42 pm #
Like Rohan said, you need a Control Sample. Get back out to Dr. Boom, fire 200 Autoshots and absolutely nothing else, and see if WWS captures them all.
When doing the Shot Rotation Tests I’d also pit a Damage Meter against WWS and see if they capture the same (or very similar) amount of damage dealt.
Other than WWS not keeping an accurate track of every shot fired, or clipped shots resulting in ammo (and arrows) disappearing from our pouches (AND quivers) I can’t think of any other explanation.
If ‘clipped’ & Interrupted shots do disappear from our Ammo pouch, all jesting about ’stolen ammo’ aside, this is a pretty serious bug for a class that needs to both buy ammunition prior to Raids AND pay repair costs afterwards. If Blizzard are inadvertently stealing Hunters’ ammunition via clipped shots (to the tune of almost 25% of our ammo) this is a very serious bug and needs to be addressed.
But maybe it’s not Blizzard (who we all know hates their bastard step-child that is the Hunter class and tries to screw us over every chance they get), maybe it really is just a flaw in WWS.
Sibohan on 08 May 2008 at 2:45 pm #
You need to manually parse your combat logs. The problem is probably WWS parsing. Remember, they had to rewrite the whole service with patch 2.4.
Tatko on 08 May 2008 at 2:48 pm #
I’ve tested on Dr Boom, both autoshot alone, and the rotation macro you’ve pasted — the combat log was 100% accurate (100 autoshots recorded in first try, 51 autoshots and 49 steady shots in the second try).
I think either you’re misinterpreting WWS, the version you’re using bugged, or you’ve made some kind of mistake in the process.
Edit: BTW, my Recount data from earlier tries had an irregularity — it seems like distracting shot didn’t use a bullet. Otherwise it was the same as combat log.
Calandris on 08 May 2008 at 2:54 pm #
Before the combat log changes I saw where the logs just didnt record everything, yes even when I was in range.
Lynx boss throws totem
totem dies
more stuff happens
totem dies
Woa where did that other totem come from, it didnt log that he threw another one. And Im afraid that I have seen it get a little worse with teh new changes. I assume that its now loggin so much that it just cant keep up and sometimes things get lost. You coudl have another party member in your group running a log and stand next to you and see if there are differences….I bet there are. Which is why 1 log will only ever be about 90-95% accuate. Or in your case 177 bullets accurate.
Kemonojin on 08 May 2008 at 3:14 pm #
I think the new combat log has some problems in general. On the Isle of Quel’Dailies, my druid noticed that when pouncing from stealth, no damage shows up in the combat log. The first thing listed is ‘melee autoattack for ###’. They are pounced, they have the little purple swirlies and don’t attack, but no damage ever shows. Ravage sometimes shows, sometimes not.
Tyns on 08 May 2008 at 3:18 pm #
Did you log out before running the WWS report? The combat log is bufferred and perhaps all of the data wasn’t written to disk before your ran the report if you didn’t /camp first.
Asara on 08 May 2008 at 4:00 pm #
I’ll do some thinking about it if you can help me figure out why my presence didn’t register as 100% on my very own log from Solarian the other night.
Rohan on 08 May 2008 at 4:26 pm #
As I see it, there are three possibilities:
1. The game itself is bugged. There’s a bug with ammo, and the game uses more ammo than shots fired.
2. The new combat log is bugged. Blizzard’s new combat log is not reporting all events.
3. WWS is bugged. All the events are in the combat log, but WWS is not picking them up.
My gut feeling is that #2 is at issue. Parsing text is not that hard, and it seems unusual to pick up some text but not all of them.
Cauthon on 08 May 2008 at 4:41 pm #
Ahh, yes, you have to exit completely because of the buffering. I bet that’s it. If this were a general problem with all hunters losing 10-30% of shots in wws, then our wws numbers would have plummeted… which they haven’t.
Azmakeron on 08 May 2008 at 5:10 pm #
I did some investigating of my own, using my own combat log from a recent Kara run. I checked my own numbers that I got from using the command line utility grep, against those WWS was showing. Here’s the numbers I found for just Auto Shot; all other abilities had pretty much the same trend.
I found that I had:
883 Hits, 192 Crits, 66 Misses
WWS found that I had:
883 Hits, 192 Crits, 60 Misses
The only difference is that WWS is not counting some misses! As Rohan said, parsing text is easy, so I can’t fathom why WWS would have trouble with misses at all, especially while it’s getting everything else correct.
Now this might explain why my total shots fired doesn’t match what WWS says, but BRK almost never misses, so this can’t be the case for him. Also WWS only miscounted 6 shots out of over 1000, or less than 0.6%. It lost around 25% of BRK’s shots!
This pretty much rules out WWS as the source of BRK’s newly discovered error in my mind. Yes WWS wasn’t prefect, but it was pretty d@mn close. I also don’t think that WoW is deleting ammo if we clip shots. If that were the case, then I think someone would have noticed it by now. That leaves the new logging system Blizzard introduced in 2.4. It might be that everything that occurs in the game is not making it into the log. But that is also hard to fathom, because why would it log one shot but not another identical one moments later.
One last thought. If after patch 2.4, 25% of all shots weren’t making it into the logs, then wouldn’t people notice a 25% drop in Damage Done and DPS? And wouldn’t they see that in things like Recount and WWS? But since no one noticed that, then maybe the logs are fine.
Edit: Just read posts that were made while I was writing this one. I agree that it is most likely buffering of the log file. Quitting WoW completely before doing anything to the log file is the best way to make sure it’s all written to disk. And that little thing about WWS not counting all our shot’s misses, let’s keep that on the down low.
Thomas Jones on 08 May 2008 at 5:45 pm #
Well…you use it, do a screenshot of it…
RECOUNT
200 bullets, look at your dps, bring up the nice little pie charts and whatnot.
Wash, rinse, repeat!
Mudge on 08 May 2008 at 5:48 pm #
@Azmakeron
I did a similar test … since I simply forgot about GREP, I loaded a 200 shot test log into Word, formatted the display so each line was complete, not wrapped. I then did a find/replace using the ‘use wildcards’ option and searched on
RANGE_DAMAGE*Auto Shot
and
SPELL_DAMAGE*Steady Shot
I got 81 Auto, 119 Steady = 200 and WWS got identical numbers.
I’m hit capped so I never miss, certainly not on Dr Boom LOL.
I agree that probably BRK is not exiting WoW completely before using the log file.
Incidently, its nice to see that my 3:2 macro is working well with my Crossbow of Relentless Strikes.
SSC on 08 May 2008 at 9:35 pm #
for the love of god, stop using 1:1 and start using the 3:2. If you are hasted enough the 3:2 becomes a 1:1. Yea thats right it auto adjusts to your haste buffs.
honestly you are geared so much better then me, but I put out a good 150 more dps then you. 3:2.
Galoheart on 08 May 2008 at 10:50 pm #
Need BRK: CSI
Waynebrady on 08 May 2008 at 11:46 pm #
I used to use a 3.2 rotation, with the weapon choices you have a 3.2 rotation might give you the most dps with your 2.9 speed weapon.
Your other weapons (2.7 speed and 2.6 speed) lend it self nicely to 1/1 though.
Why not just get the new xbow badge and use a 3.2 rotation
willstream on 09 May 2008 at 12:01 am #
I’m going to vote for the combat log…
I currently have it filtered to “Self”
Yet I keep seeing strange things like “Character Creates Spiritual Soup”
Character being some random player I don’t know…
Of course I was in Thrallmar at the time and as those who do the Cooking daily know… you can only make Spiritual soup at the Ancestral Orc burial grounds at the SW corner of Nagrand.
To me this says that the log is dropping/missing/adding and/or mis-interpreting data a great deal
It'h a conspirathy on 09 May 2008 at 7:21 am #
Cia confiscated your ammo without consent to support the ongoing war effort in Iraq, then purged your logs.
Mudge on 09 May 2008 at 8:14 am #
@willstream
If the log is wrong it must be set up wrong … I have now done a dozen more tests and the log and WWS always agree on number of shots. I plan to post a web site discussing my results which compare the Wolfslayer Rifle with the Relentless Strikes Crossbow.
Note: It is clear that more than 200 shots will be neccessary to get a statistical significant comparison. With 1000 shots I get a number like 600 dps (no bufs except AotV) with a one sigma of about 10. 68.2% of tests will fall within one sigma (so 31.8% will fall outside this range). The numbers for higher sigmas are: 95.4% (”two sigma”), 99.7% (”three sigma”) . So you need 2 sigma to get to the point where only 1 test in 20 will fall outside the +/- 2 sigma range. That is to say, 19 out of 20 tests will lie in the range 600 +/- 20.
Research in the physical sciences usually expect 3 sigma (only 3 data sets in a 1000 would fall outside the range).
Rampel on 09 May 2008 at 9:13 am #
I would not be surprised if the combat log is broken.
Even before the 2.4 patch I had noticed irregularities.
Thus, I don’t trust damage meters for anything resembling accurate results any longer.
Rotseh on 09 May 2008 at 11:53 am #
I’m sure it’s because of the buffer mechanism of the combat log.
Since initiating and completing disk writeouts takes some time, log data is buffered in memory to be written to disk less frequently in bigger chunks. This saves time but is prone to data loss if one does not log out or if the game client crashes.
Kitara on 09 May 2008 at 12:00 pm #
castsequence macros are not used as much as /cast /cast macros these days. Unfortunately I do not know the current form of the two part macro.
Vurag on 09 May 2008 at 12:28 pm #
Use Recount, for the love of all that is huntery!
Irshi on 09 May 2008 at 6:34 pm #
I’d like to echo a few of the suggested changes to your tests.
Make certain your log is switched to ’self.’ It should only display things that happen to you or that you do. This will limit the clutter that may cause it to miss shots. (Dr. Boom tosses a Boom Bot …)
Use a real time DPS meter and WWS. Compaire the results. You can probably save yourself some time by dividing the log using WWS’s tools as opposed to logging out and back in after each test but depends on your login/logoff speed and how well you know the tools.
Use time instead of ammo. This will save you from having to run back to the bank. It’s also more realistic to how we actually fight. You don’t fire 200 or 1,000 shots at a boss. You fight him/her for x minutes. I personally use 2m increments for my tests as it allows me to use my CD’s (if I so desire) for each test.
Try multiple rotations. Use a 3:2 rotation for your slower weapon (Tuskbreaker) and see if it does more damage.
I think you’re likely to see BBLR outdpsing all others with a 1:1 rotation. BBLR is the best drop pre-T6 stuff, possibly better than Vashj’s bow with a 3:2 rotation. (Speaking strickly for BM hunters)
Theaah on 10 May 2008 at 7:11 am #
I know the gun racial bonus is nice and all, but the Crossbow of Relentless Strikes really is the best BM weapon in the game. I would highly recommend getting it. (I suppose the bow from Sunwell might be better [I haven't plugged that into cheeky's to find out], but that’s about it save for the Legendary bow of course. And the bow from Sunwell might not be due to the speed on it.) It’s higher DPS than anything you’ll find before BT/MH and it’s the proper speed for BM. (Which honestly makes it better than the BT/MH ones even though they’re slightly higher DPS; the speed on them sucks.)
I can understand being attached to your weapon. I used Don Santos’ for AGES and it was really hard to actually part with it for the Chicken of Relentless Strikes. (Have you looked at the thing? It’s ridiculous looking.) But once I started using Crossbow, I fell in love. It’s a absolutely beautiful speed (granted I don’t have much passive haste gear) for manually threading, even with Quick Shots procs. And, of course, lovely to use with a 1:1 macro. (I manually weave when soloing and farming and use a macro in instances.)
Schadenfreud on 10 May 2008 at 12:40 pm #
I’ve been doing a bit of reading up on this. And I think those shots might have been blocked.
As discussed here on the EJ forums. http://elitistjerks.com/f47/t15685-ranged_combat_mechanics/
BTW, the new crossbow is awesome, and I highly reccommend for any BM hunter.
Schadenfreud and Yossarian
Noxveritas and Ripley on 10 May 2008 at 3:47 pm #
It would be fairly easy to test for blocks. go jump up on the spikes behind him or perhaps the barrels to his side and plink away from over there. Blocks only occur from in front.
Still would not explain why all other hunter’s do not report the same problem. I think I may test it myself a touch today.
Q on 10 May 2008 at 5:02 pm #
Dr. Boom will turn to look towards you even if you shoot / cast from the side, so attacking him from the rear will not remove the block option from this hit table. Nice thought though
For re-testing purposes, I’d suggest using a far smaller sample, say 20 arrows, using recount and WWS against each other, along with your combat log, maybe a pencil and paper, and lather, rinse, repeat until you find which shots appear to be getting missed.
As for blocks, I’m not thinking that Dr. Boom has a 15 - 30% block rate, but I could be wrong
Irshi on 10 May 2008 at 5:09 pm #
Blocks, in WoW, are only partial. So if the shots were blocked it’d show up in the log as mitigated. They’re still hits.
A blocked shot looks like this in the logs:
You hit mob X for 394 (394 blocked)
Felandra on 10 May 2008 at 8:43 pm #
30% Blocks? Highly unlikely.
Also, BRK has told us why he won’t be using the Chicken-on-a-stick, and that is fine. Guns are just way more satisfying overall. And since BRK hates rotation macros with a passion, I doubt we will ever see him drop the badges and get a crossbow he hates using a macro he hates. No point in posting only to tell him he needs the crossbow.
Just drop it.
Anyway you are all wrong in regards to determining the DPS of each weapon. No damage meter addon or even WWS can truly help you. They are all affected by luck and chance. And like in math you can do a million testruns, you will still not have performed the correct empirical test. You have to do the theory to find that.
Simply, go to Boom or your testsubject of preferance, shoot for a set period of time with each weapon (and rotation of course), make sure you do your best to avoid clipping but still stack it as much as you can. Basically do your utmost on each test. We have to assume that you can do the same most of the time.
Then check up on Recount or whatever other addon you might have, how many shots you fired of each type. Then CALCULATE the average damage for each shot. That can be a tough one, granted, so I suggest getting the addon called Dr Damage, it is your own little pocket theorycrafter in regards to damage per shot. Then just add it all up and devide with the amount of time you used in the tests.
Only THEN will you find a solid result. One that isn’t marred by too many crits or low/high rolls on the damagerange.
I have the crossbow and such a test showed that a 3:2 rotation beat a 1:1 with about 60 DPS (no armor included though). That’s considerable, and enough to let me use it during high mana raids.
Schadenfreud on 11 May 2008 at 6:08 am #
I went to test on Dr Boom last night. And you’re right he shouldn’t be blocking any shots from a decently geared hunter.
I will be counting my arrows on my next raid though.
BRK, of course, can make up his own mind on which weapon he prefers. I suggest that other hunters who are interested in MQOSRDPS should look into the crossbow, since I believe it does more damage.
Don’t take my word for it though, I’d highly suggest making up your own mind.
Rj and Tito on 13 May 2008 at 12:23 pm #
2. The new combat log is bugged. Blizzard’s new combat log is not reporting all events.
Not only is it not reporting all events, it’s reporting weird events. I’m in Terokkar Forest tracking Ironjaw solo, and I’m getting reports on my combat log from people fighting… Erratic Sentries? Heywaitwhat??? Aren’t they on QDaily Isle… on an entirely different planet??
I like the idea of the new combat log, but it is obviously borked.