> You are correct the data is all CURRENT. I do not have historical data,
> but I grant that their work has improved over the years. But rather than
> "learn" from looking at a comingling of all data, they have
> "learned" by considering different materials and techniques
> individually.

Hmm. Perhaps we are saying the same thing, but doing so in such a manner as to confuse the other. I'm going to use a lot of words now to describe my thinking, and we'll see if you think it matches yours.

Let me use the example of the long running studies B&D has, was, and still is doing on saw blades. All the data is kept and is available. Tests themselves are done to a standard that was set some years ago.

When a new batch of saw blades comes in for testing we do the test just as we've done for years. Same particle board, same motor, same same same. Now, when I compile the data of the test it is only that data that is compiled.

To make it clearer, lets assume I've yet again received another batch of 10 Dewalt thin kerf 32 tooth blades. Now I'll test these alone, and compile their data. But I would not take last years data from the exact same blades and mix them in and extrapolate an average.

What I might do, if requested, is compare (not mix, but compare) last years data for those saw blades.

Now in doing this test it obviously is going to take me a while. Probably about two weeks. If nothing changed in the test, I will mix the data for results. As in averages, regression etc. It's only when something changes that it becomes significant. Such as changing saw motors or using a different type of material.

In the progression of creating that DeWalt thin kerf 32 tooth blade, I will use old data for progress comparison. One can not look at todays cut time of 2.3 seconds and pronounce the saw blade improved. Only after looking at the previous version and its cut time of 2.8 seconds can it be pronounced improved. And the testing procedures for both must be identical. If the motor was changed, or even if significant maintenance was performed (changing motor bearings, rewiring the shop, etc) then a strong conclusion of better cannot be extrapolated from the data.

However, some data can be extrapolated. Records are kept for this purpose. Lets assume the bearings were changed on the motor. Because of the records on the machinery and testing results I can see that a change in the bearings resulted in a change of approximately .2 seconds per cut. I can then quite easily include that information in the test results, and show that the new saw blade did not simply go from 2.8 to 2.3 seconds, but that a new saw blade and new bearings went from 2.8 to 2.3 seconds. And that the bearing change accounted for .2 second of that change. So the delta t becomes 0.3.

Now, if I do happen to find an even older test, lets say a very early preliminary test, that was performed with a different motor, different material, with different procedures and different criteria, that data cannot be interpolated into the current test. The most that can be done with it typically is to foot note it. This is also why consistency is far more important in testing then sense. A test may have nothing to do with reality, but as long as they are done consistently, they do allow comparison. Without consistency of the test (not the product) comparison cannot be made.

To wit, you cannot compare the ability of chewing gum to drill through cement with rate a lollypop disolves in gasoline. But you can compare the ability of chewing gum to cut glass to the ability of a lollypop to cut glass.

> If we look at the composite data, we can agree that for each vendor on
> each layup we get a different bell shaped curve. I said that if you
> combine the data for all the vendors on each layup individually you no
> longer get a bell shaped curve - it is modal with a peak for each vendor.
> A look at the data should show that.

I think you are misunderstanding the bell curve. That or misapplying it. Lets go back to my DeWalt saw blade testing. A saw blade cuts best when new, and worse as it ages. The cutting time, required force, and electrical consumption steadily increase. There is no bell curve here. There is also no bell curve with breaking composite boards.

Where there is a type of bell curve is when I compare a sample pool of 10 identical saw blades in doing this test. When I compare the time and force required at the begining of the test, there is a bell curve there. With most of the blades performing in say 2 seconds. I'd get a bell curve (roughly) of 2 with a deviation of 0.3.

Now, when the first saw blade experiments were performed that bell curve would have been very wide. With perhaps an idential mean time of 2 seconds, but with a deviation of say 8.7 seconds. As the product was refined and improved, the bell curve (and deviation) would tighten up. That is the normal result of refinement.

Lets also suppose I had several competitors saw blades in that test. Where would they fit within the DeWalt curve? No where. For they are completely different. While they are tested the same way, their results are singular (for each competitors product), and are used for comparison evaluation.

Lets say Freid has a new 32 tooth thin kerf blade. It cuts with an initial time of 1.8 seconds, deviation of 0.2. That does not change the values of the DeWalt tests. It cannot. What it does tell us is that this sample of Fried blades did better then this sample of DeWalt blades on initial cut times for idential loads through idential materials with idential parameters. And absolutely nothing else.

Another reason to compile data done the same way is to reduce the deviation and improve accuracy. We take those DeWalt blades, test two of them and one of them breaks. We now have a failure rate of 50%. That is nota good rate. But is it a real rate? We continue testing over the next three years. We test 10,000 blades, and have three break. Now the failure rate is 0.0003%. Not only is that rate far better, but it is far more accurate. Simply because the sample population is larger.

> I said that no matter how many results any of the vendors submits for a
> LAYUP they will not move their bell shaped curve for that LAYUP. If they
> could "learn" I would expect that the ranking of vendors within
> a layup would be random across the various layups. It is not.

I do not understand at all why you think product improvement through learning would result in random results. That is absurd on the face of it.

> Consider testing polyester resins 20 years ago and epoxy resins today. If
> you combine your data prior to making conclusions, that will be worthless.
> If you compare your conclusions from the individual data sets, that will
> be worth while.

As I have said, and shown, one does not simply gather all history and average the results into a large vat of useless numbers. One uses the older data of 20 years ago to determine if todays product has improved or degraded with the current refinements. Only if one is interested in the 20 year historical strength of polyester resins would one agretate the historical data.

> The composition of epoxies varies at the manufacturer's whim. If a large
> user reports problems, the epoxy may change. Unless you are notified of
> the change, you may be unable to distinguish between "learning"
> and changes in the product.

If it is done purely on whim, that would be correct. However, I seriously doubt it is done in such a whimsical fashon. End users who depend on the product to have a given set of characteristics would not be able to depend on the product. Also, in todays regulatory environment, there are many restrictions and expences incurred with changing a product formulation, particularly one so closely regulated as the resin industries are.

> I have made a lot of mistakes in the past. I have learned so much from
> them that I am a different builder now than I was even last year.

Exactly. And if you keep record of the changes you've made you can determine what changes helped, what hurt, and guide yourself to better improvements next year. Otherwise, you are blindly foundering and only improved by luck.