Help me get stats on lenses problems

[Mantat]

Hi all,

I made a website dedicated to photographic equipment informations and buy/sell. It just added a new feature that allow user to vote if they ever experienced any of the common issues with their lenses (ex: soft, fungus, etc..). My goal is to gather some real statistics about these problems so buyers can have a clear picture of the situation.

Basically, to vote for a problem, a user must have that specific item in his possession list (gear list tab for registered user). This will hopefully allow us to have stats about people who actually used the lens, and not hear-say....

Right now only the Canon gear list is in the database but I will ad Nikon as soon as I have enough time.

Site URL: http://megapixelicio.us

Let the math rules over the rumors!

Note: site is still in beta (will be until Nikon stuff is in) so please, if you have any feedback, PM me!

 

[TimJim]

Sorry i can't help with the stats, but i have to say your site looks very nice. I would add pictures of the gear after you click on the link so people would know what it looked like.

 

[cube]

http://www.sveido.com/lensdb/main.asp

 

[Mantat]

Sorry i can't help with the stats, but i have to say your site looks very nice. I would add pictures of the gear after you click on the link so people would know what it looked like.

Of course you can help! if you have a canon SLR and a few lens, you are exactly what I need! ;-)

As for the pictures, I wanted to do it, but the problem is getting pics of each piece of gear, they are under copyright by Canon... But eventually, if I can get users to submit good pictures of their gear, I would use these.

Like that one from my D5 (taken with a G9)

 

[Mantat]

http://www.sveido.com/lensdb/main.asp

Nice! But I think my implementation of the solution is better and use a more statistical approach.

 

[TimJim]

Of course you can help! if you have a canon SLR and a few lens, you are exactly what I need! ;-)

As for the pictures, I wanted to do it, but the problem is getting pics of each piece of gear, they are under copyright by Canon... But eventually, if I can get users to submit good pictures of their gear, I would use these.

Like that one from my D5 (taken with a G9)

I don't use Canon sorry, but user pictures is a good idea.

Also some sort of area where users can chat or share information, pictures, how to's, etc. would be awesome. Possibly implementing a small forum or something.

 

[cube]

Nice! But I think my implementation of the solution is better and use a more statistical approach.

The point is that you can see there many detailed problems, not a simplistic approach.

 

[compuwar]

Nice! But I think my implementation of the solution is better and use a more statistical approach.

It's my understanding that self-selected random sampling is generally considered statistically invalid until you get to a very large portion of a particular population.

Lumping performance issues that are based upon the optical design in along with environmental differences like fungal growth seems sub-optimal to me, any normal lens in the wrong environment will get fungal growth, and it may actually be that some lenses end up on that list because they're better suited for harsh environments and simply end up there more frequently because of that- that would artificially taint a lens that's good in harsh environments which hasn't been in a harsh environment.

Finally, many manufacturer's made the same lenses over a long period of time, and anyone who's been involved with any sort of manufacturing will know that even for products who's life spans only a couple of years, parts suppliers may change parts, so mechanical defects may be due to a particular item being in a series (like Powerbooks and battery problems) where a sub component's source was different- making overall statistics poor indicators for a particular sample even if the statistics were statistically valid.

I'm not a fan of the approach as described.

As far as adding lenses for Nikons, there are about 400 F-mount lenses- it'll be interesting to see what selection criteria you'll use to decide which lenses to include

http://en.wikipedia.org/wiki/Nikon_F-mount#Compatible_Lenses

 

[GoCubsGo]

It's my understanding that self-selected random sampling is generally considered statistically invalid until you get to a very large portion of a particular population....

SNIP SNIP

At first glance I immediately thought of MacRumors because I stumbled on a post where someone said if there weren't over-paranoid people who bitched about their new computers then this site would be a morgue. Similar to a new computer it would seem to me that you will have a weighted average in favor of the bad coming from a site like this. While I think overall it is nice to have the internet to go out and research a product that you're in the market for, basing a buying decision 100% on what is found online is difficult.

People often come back and point out the bad before the good and sometimes the good isn't offered up at all. Aside from the statistical data you're willing to offer from an uncontrolled population, I think the site can be taken more seriously than say some newbie ranting about a "mooing" in their computer, but not seriously enough to not attempt to seek out more research.

Overall the site is nice and I admit to not understanding the method in which you are choosing lenses, especially for Nikon. I think the site can and will contribute to the research of buyers, but as with anything else online, it has to be discounted to a certain degree because of the simple fact that people complain before they cheer for a product all too often.

 

[Mantat]

Hi,

first thanks for these two last replies, I gave long thought about these points and I agree that people are quicker to point out flaws but at the same time, since every user of the site will eventually fill that survey, I think it will even out in the long run.

It's my understanding that self-selected random sampling is generally considered statistically invalid until you get to a very large portion of a particular population.

It depends of the distribution of the results (normal law, Fish law, etc..). I have taken master level courses in stats (yes, I am sorry too!), so I know the limit of these sampling methods but that is the best I can get. Also, I am not looking at hard facts, but at the perception of the issues from their users.

Also, I dont want to explain the whole details of how I am going to display the stats in the end, but be sure that it is much more complicated than just making averages of the results. But at the same time, I am not going to rig the results to take into account any psychological preferences. Once the sample is big enough, I will start detailing on these subjects.

As for the Nikon, it is simple... My goal is to enter every single lens ever made that fits on a Nikon body. And yes, I know how big of a task that is! This is why I did Canon first. Nikon is next, or maybe Sony (Minolta)... I have to think about it first because that is going to take a few 100 hours since I enter a lot of data that is not yet displayed on the site.

Thanks again for the feedback!

 

[cube]

You have to match lenses to bodies, as there are issues that won't appear everywhere.

 

[Mantat]

You have to match lenses to bodies, as there are issues that won't appear everywhere.

I know, but that is not my original intend. I might add a special 'warning' section later about this. The idea so far is to capture the chances of production defects or random flaws in some models.

Issues identification are only one part of the site, so I dont want to go too deep it in at the expense of the other areas. But if users ask for this, I will definitely add it. Right now my concern is more about adding another Brand of products.

 

[ChrisA]

My goal is to gather some real statistics about these problems so buyers can have a clear picture of the situation.

It won't work. Well it might collect information but the statistics will not be valid. You problem is that your samples will be biased. T do this right you would need to randomly select owners and asked them some standardized questions. Getting a good random sample of users is the hardest thing.

The other thing working against you are the numbers of people you will need. I doubt even one in a thousand lenses have problems. So to get statistically significant numbers of many lens issues you will need many hundreds of thousands of people to enter data. That is not going to happen

 

[Mantat]

It won't work. Well it might collect information but the statistics will not be valid. You problem is that your samples will be biased. T do this right you would need to randomly select owners and asked them some standardized questions. Getting a good random sample of users is the hardest thing.

The other thing working against you are the numbers of people you will need. I doubt even one in a thousand lenses have problems. So to get statistically significant numbers of many lens issues you will need many hundreds of thousands of people to enter data. That is not going to happen

Random selection: I agree, but at the same time it depends of what I want to measure. I am not trying to get the real stats of the % of defective products coming out of the factory. I want to know the stats of the perception of the visitor of the site. Visitor of the site, might not be a valid sampling of the general population, agreed, but they are a better sampling than the complainers in the forums. And just that is an improvement over the current situation. And that is what I want to achieve.

As for your other point, my bet is that the variation is bigger than what we should expect. Just look at people doing rigorous testing of lens and you can see variations in focussing/sharpness over 3-4 lenses bought at the same time! I am pretty sure that some of the issues are much more rare than others (or non existent) but what I want to show in this is if one lens is more subject to a specific problem than another, not the % of soft lenses for, let say, a good and trusty lens like the 50mm f1.8.

I know the approach is not perfect, but it is the best that can be done under the current situation so I will have to manage with it for now. If my sample if big enough one day, I will do statistical analysis of the sample itself to see any pattern.

 

[scotthayes]

Hi all,

I made a website dedicated to photographic equipment informations and buy/sell. It just added a new feature that allow user to vote if they ever experienced any of the common issues with their lenses (ex: soft, fungus, etc..). My goal is to gather some real statistics about these problems so buyers can have a clear picture of the situation.

Basically, to vote for a problem, a user must have that specific item in his possession list (gear list tab for registered user). This will hopefully allow us to have stats about people who actually used the lens, and not hear-say....

Right now only the Canon gear list is in the database but I will ad Nikon as soon as I have enough time.

Site URL: http://megapixelicio.us

Let the math rules over the rumors!

Note: site is still in beta (will be until Nikon stuff is in) so please, if you have any feedback, PM me!

I've registered and added my kit to my profile, not sure what you need or want doing next.

 

[Abstract]

I think the problem is going to be what ChrisA said, which is getting enough people who (1) Have a problem with a lens, and (2) Getting that person to go to your site and enter this information.

If the average person who visits your site has 2 lenses, and only one 1 out of 10 people have a problem with one of their lenses, then you're going to have very few comments per lens. There's a lot of lenses out there, and you're going to get 1 problem per lens. For some lenses, you're not going to see any response.

It's probably better if you just had 7-10 different criteria for lenses, and ask each user to fill in a review, whether the lens was good or bad. That way, it won't be (as) biased. You end up with some biased users who gush over how great their lens is because of several great photos they took 2 years ago (that could have been taken with any similar lens from a different brand), and you're going to get dissatisfied people who want to bitch and moan. That's 2 extremes that will even themselves out. Of course, you'll get a decent percentage of the population who would give you a balanced review.

 

[Mantat]

Hi thanks for the feedback,

but I just want to point out that the know issues section is not only for people who are having problems with their lenses, it is also for people who never had any issues.

That being said, I have to agree that people with issues are most likely to fill the page at first. But I think that in the long run it will even out. It is exactly as in camera/lenses review forums, for each positive review you get 20 bad because people dont see a reason for wasting energy to be positive while it is liberating to spit the bad things. My bet is that it is going to be so easy for them to give their assessment that they will do it, good or bad.

Maybe I am wrong, time will tell. But as I said previously, this is just one of the many features of the site and as time pass by, you will see links between each of these functionalities.

I think the problem is going to be what ChrisA said, which is getting enough people who (1) Have a problem with a lens, and (2) Getting that person to go to your site and enter this information.

If the average person who visits your site has 2 lenses, and only one 1 out of 10 people have a problem with one of their lenses, then you're going to have very few comments per lens. There's a lot of lenses out there, and you're going to get 1 problem per lens. For some lenses, you're not going to see any response.

It's probably better if you just had 7-10 different criteria for lenses, and ask each user to fill in a review, whether the lens was good or bad. That way, it won't be (as) biased. You end up with some biased users who gush over how great their lens is because of several great photos they took 2 years ago (that could have been taken with any similar lens from a different brand), and you're going to get dissatisfied people who want to bitch and moan. That's 2 extremes that will even themselves out. Of course, you'll get a decent percentage of the population who would give you a balanced review.

 

[compuwar]

It is exactly as in camera/lenses review forums, for each positive review you get 20 bad because people dont see a reason for wasting energy to be positive while it is liberating to spit the bad things. My bet is that it is going to be so easy for them to give their assessment that they will do it, good or bad.

How do you plan to deal with astroturfing? There are some brand fans who'll say anything good about anything by a particular manufacturer and you'll probably find that the sampling gets even worse for the small-share users who can be easily swayed and disproportionally represented.

 

[Mantat]

How do you plan to deal with astroturfing? There are some brand fans who'll say anything good about anything by a particular manufacturer and you'll probably find that the sampling gets even worse for the small-share users who can be easily swayed and disproportionally represented.

I dont think that fanboy would be an issue here because these are not reviews but realization of problems. Even fanboys are willing to complain when something is not as good as expected. Just look at all the people complaining about bugs with their macs. I would not think that they would be so in love with, let say, canon that they would deny that their lens is not sharp.

Of course, I might be wrong, time will tell! Right now my problem has more to do with getting people to register and vote!