Interesting results?

[halocline]

Now I understand the tests; the numbers are identical and represent the load and depth. The dots represent the score, so the G260 really did do quite a bit better. I guess it doesn't surprise me too much. I wonder if exhaust effort is part of the score; if so, it could explain why the G260 does better.

Not that it matters. Regulator performance while diving is remarkably subjective IMO. I just evaluate them on how they feel. Among my favorites is the balanced/adjustable, which likely performs relatively poorly on these tests. The lack of an aggressive venturi assist combined with a warmer moister air just makes the reg more enjoyable to me.

Interesting that the A700 is an attempt to bring back the benefits of a metal case, but it shortchanges those benefits by being low volume. The lower volume means the air spends less time in the case, and as such gets warmed less and condenses less.

SP used to be pretty innovative with 2nd stages. Between the 109, pilot, G250, and D series they really had something going on. A couple of dirt simple workhorses and a couple of design-nerd creations that actually had substantive benefits. Apparently those days are gone.

 

[redacted]

According to the scubalab page, this is how the tests were conducted:


Seems like a reasonable attempt to compare equipment to me. The linked screenshots only include breathing simulator results and omit subjective "ergo" results.

I'm surprised the Posieden didn't do better, and disappointed that they tested such a limited variety of regulators. I've been eyeing Atomic and Posieden for some time now.

---------- Post added November 16th, 2015 at 11:10 PM ----------

Actually, I can't find the subjective score anywhere. Maybe they forgot to include it!

There are some rather important considerations that are not mentioned in their test design.

How many of each item are tested? One is not a very good answer as there may be item to item variances. Three is usually a minimally acceptable number with a process for handling high item to item variance results.

What preparation does each item undergo before testing? It would probably be best to insure each item is tuned to some acceptable/standard specification if the intent is to present results as representative of the population. One might argue that it should be tested "out of the box" or "as delivered" but that may just introduce another uncontrolled source of variance and reduce the value of a sample as a predictor of the population performance.

What is the scale of the results? What does each increment in the scale represent? I suspect there presentations are pass/fail for each of the 4 test conditions which really does not scale. Somewhere the results user should be able to access the underlying quantitative results. A Posiden may well have had the best quantitative results for recreational depths but you can't see that with the way the results are presented.

 

[kelemvor]

There are some rather important considerations that are not mentioned in their test design.

How many of each item are tested? One is not a very good answer as there may be item to item variances. Three is usually a minimally acceptable number with a process for handling high item to item variance results.

What preparation does each item undergo before testing? It would probably be best to insure each item is tuned to some acceptable/standard specification if the intent is to present results as representative of the population. One might argue that it should be tested "out of the box" or "as delivered" but that may just introduce another uncontrolled source of variance and reduce the value of a sample as a predictor of the population performance.

What is the scale of the results? What does each increment in the scale represent? I suspect there presentations are pass/fail for each of the 4 test conditions which really does not scale. Somewhere the results user should be able to access the underlying quantitative results. A Posiden may well have had the best quantitative results for recreational depths but you can't see that with the way the results are presented.

For the objective test, it's not a pass/fail. The document clearly indicates they measured work of breathing with a test devised by the USN. Score ranges were assigned numbers from 1-5, and the results of the test go cleanly into those results. You must have missed it in my earlier post:

This is the U.S. Navy’s Class A test depth and breathing rate (although the Navy uses a higher HP supply pressure than we do). The simulator monitors how much effort is required to breathe, measuring the work of breathing in joules per liter (j/l). In our ratings, a score of 1=3j/l or greater; 2 = 2.26-3.0 j/l; 3 = 1.51-2.25 j/l; 4 = 1.1-1.50 j/l; and 5=1j/l or less.

In every other case where products are tested that I know anything about it's typically just one. The sponsor sends the tester whatever is to be tested. If it needs to be tuned, the sponsor should tune it before sending... or possibly suffer a bad review. AMD is known for screwing that part up with video card reviews. If the manufacturer that worried about it, they could always send a tech along with the product to ensure optimal performance. That happens, I've read about it in review articles of some products. This is even the case for cars and supercars. Recently Lamborghini provided a single Huracán to ars technica for review. Ars wrote a fairly scathing review of the car, and Lamborghini had to live with it.


Think about it this way. If the manufacturer can't trust a single product to be of a standard enough quality for review.. how in the world could a consumer have any confidence the product they buy in a dive shop is going to work correctly?

Since there isn't anything better out there that I know of, I think this would be valuable if they had tested regulators I was considering. If scubapro doesn't like the results of a test that a "journalist" performed, they should call and send a new unit, and ask the journalist to retest. Maybe they will. It seems like it would be easy to do since it's an online publication.

 

[redacted]

Yes, I did not pay enough attention. But that makes me even more suspect of those results since I am pretty sure the performance differences between an A700 and G260 are minimal. When the results are hard to believe, I have to ask myself how this could happen.

One obvious possibility is they are not testing a representative sample or that their sample is, somehow, deficient. It is a bad tester who lets sampling errors bias results that are presented as representative of the population. If a major variable in this sample was how well 2 items were tuned, out of the box, what value is that test in making a purchase decision?

I have seen enough questionable results before from Scubalab before (regulators that differed only by the name on the cover showing different performance results) to not put much weight in their test results.

When the USN performed similar tests, they tuned each regulator to spec before testing it. I don't recall, but I am fairly sure they also tested multiple samples of each item. Single sample testing is just not very useful beyond the prototype development stage and probably only found in production item testing when samples are prohibitively expensive and the tests are destructive (like atomic bombs).

Scubapro may actually like those test results. It does look real good for their more popular and cheaper to make G260.

 

[scubafanatic]

Yes, I did not pay enough attention. But that makes me even more suspect of those results since I am pretty sure the performance differences between an A700 and G260 are minimal. When the results are hard to believe, I have to ask myself how this could happen.

One obvious possibility is they are not testing a representative sample or that their sample is, somehow, deficient. It is a bad tester who lets sampling errors bias results that are presented as representative of the population. If a major variable in this sample was how well 2 items were tuned, out of the box, what value is that test in making a purchase decision?

I have seen enough questionable results before from Scubalab before (regulators that differed only by the name on the cover showing different performance results) to not put much weight in their test results.

When the USN performed similar tests, they tuned each regulator to spec before testing it. I don't recall, but I am fairly sure they also tested multiple samples of each item. Single sample testing is just not very useful beyond the prototype development stage and probably only found in production item testing when samples are prohibitively expensive and the tests are destructive (like atomic bombs).

Scubapro may actually like those test results. It does look real good for their more popular and cheaper to make G260.

Are we sure SP likes the way the 'cheaper' G260 embarrassed the more 'spendy' A700 ? I suspect the manufacturing cost differential is substantially less than the profit differential between the units.

 

[buddhasummer]

I'm sure they did a test when the A700 first came out and I'm sure they at that time gave it perfect scores across the board. I'll see if I can find it. Maybe the black reduces performance, perhaps tactical comes at a cost.

---------- Post added November 17th, 2015 at 08:44 PM ----------

Apparently they changed their mind. Doesn't do much to instill any faith in their reviews:

60:Second ScubaLab - SCUBAPRO MK25/A700 Regulator | Scuba Diving

---------- Post added November 17th, 2015 at 08:46 PM ----------

And this:

SCUBAPRO MK25/A700 | Scuba Diving

---------- Post added November 17th, 2015 at 09:03 PM ----------

Conclusion, wonky results or it's all just nonsense.

 

[Centrals]

Scubapro may actually like those test results. It does look real good for their more popular and cheaper to make G260.

Not sure about this.
The general perception is expensive = better if it is from the same manufacturer.

 

[skippy11v2]

For the objective test, it's not a pass/fail. The document clearly indicates they measured work of breathing with a test devised by the USN. Score ranges were assigned numbers from 1-5, and the results of the test go cleanly into those results. You must have missed it in my earlier post


In every other case where products are tested that I know anything about it's typically just one. The sponsor sends the tester whatever is to be tested. If it needs to be tuned, the sponsor should tune it before sending... or possibly suffer a bad review. AMD is known for screwing that part up with video card reviews. If the manufacturer that worried about it, they could always send a tech along with the product to ensure optimal performance. That happens, I've read about it in review articles of some products. This is even the case for cars and supercars. Recently Lamborghini provided a single Huracán to ars technica for review. Ars wrote a fairly scathing review of the car, and Lamborghini had to live with it.


Think about it this way. If the manufacturer can't trust a single product to be of a standard enough quality for review.. how in the world could a consumer have any confidence the product they buy in a dive shop is going to work correctly?

Since there isn't anything better out there that I know of, I think this would be valuable if they had tested regulators I was considering. If scubapro doesn't like the results of a test that a "journalist" performed, they should call and send a new unit, and ask the journalist to retest. Maybe they will. It seems like it would be easy to do since it's an online publication.

ScubaPro was listed on the Navy's AMU list with a MK25/S600 for warm water diving a few years ago, they are no longer listed. Why use a test methodology for an organization that has a approved listing?

I will say when I have to set up a new reg for a customer, ScubaPro typically requires no adjustment, where some other manufacturers actually tell their certified technicians that it is their job to make sure that the regs out of the box are within specs for them.

Btw, you can find the AMU list here.. http://www.supsalv.org/pdf/AMU110315.pdf

 

[BDSC]

The very best regulator out there is the Atomic T3. Why? Cause I own one and paid a lot of money for it so it has to be the best. How do you like that for some solid evidence.

 

[scubafanatic]

The very best regulator out there is the Atomic T3. Why? Cause I own one and paid a lot of money for it so it has to be the best. How do you like that for some solid evidence.

...but have you installed an Apollo Bil Filter soaked in vanilla extract into the primary 2nd stage hose line ? YUMMY!