How do you test new gear?

[Darmok N Jalad]

So I just ordered a new telephoto lens, the Panasonic 100-300 mkii. I looked through several pages of sample shots on a m43 forum before I committed to buy it, and now that it has arrived, I wanted to put it through its paces to see that I got a "good sample" lens, and also to find the practical limits of the gear. It got me thinking, does everyone else do something like this, or do you just go out and shoot?

For example, I just did a test were I stand up 3 playing cards and then took handheld shots at various ISOs adjusting shutter accordingly. That way I can see how low I can go with ISO and shutter speed before things get messy, and also to see how clear the results are, edge to edge. I guess if nothing else, it helps me figure out why I might get a poor result--is it a bad setup, or am I simply asking too much of the gear?

Thoughts?

 

[mollyc]

I just shoot. In general I know my camera body (unless it’s the “new” in question). It won’t let take me many frames to know if something is off.

I’ve had one lens I sent back to b&h and one other I should have but kept (and later sold). Otherwise I’ve been happy with everything I’ve bought. I can tell within just a few frames whether something works right or not.

 

[someoldguy]

I get a lot of glass used so I always do a test on them . Basically , I set the camera and lens on a tripod , use a remote , and take 2 sets of shots , 1 wide open and 1 at f5.6 or 8 at each of the focal lengths indicated on the lens barrel .I'll shoot RAW & JPEG . One set is at infinity , and the other at close range .The shots are taken around mid day at a local historic site so there's some side lighting in one of the upper corners and lots of detail to pick around . I set the tripod up at about the same location every time. When I get home , I'll pixel peep the images at 100-150% and see what I find.
Usually , there's no surprises in the results , but sometimes stuff that's highly regarded proves to be less so , and sometimes lesser glass shines .
After I get this done , I figure I know what the lens will do , and then I'll just use it keeping in mind the test results . Not scientific , but works for me.

 

[Clix Pix]

I just start using the new gear! Of course in this most recent case for me, everything was new, unlike in the past when I would, say, buy a new body or a new lens and already have the other gear. This time it's been a learning curve with both camera body and lenses simultaneously. As Molly has mentioned, I know when something isn't right, either with the camera body or the lens. Especially in the beginning with new gear I also realize that there is likely to be user error or some fumbling around with, "now how do I --??" until I am totally accustomed to it. It helps with mirrorless having the EVF as one can see immediately what the scene will look like in terms of exposure values, etc. In the past I can recall shooting something, then chimping (or worse, waiting until I got home to look at anything) and realizing that oops, I'd used the wrong exposure values! It's helpful when making adjustments with the camera held at one's eye to see what's happening immediately, but also helps establish "muscle memory" about the buttons and dials as well.

With the new gear, since I already had other Sony gear in the past, although not a full-frame mirrorless body before, I at least had some familiarity with how things should work and the often-complained-about setup menu wasn't too bad to get through, either. That said, I don't always set up a lot of custom functions with some buttons, the way others might choose to do.

With a lens, if I'm looking at the results in the computer and something doesn't seem right on a consistent basis, if there is an issue with back-focusing or front-focusing, for example, I'd spot that pretty quickly after the first few uses of the camera and lens. I've been fortunate in both my Nikon gear and now my Sony gear that I have not had issues. I buy from local retailers so that if there are problems it is easy enough to get in the car and take the problematic item back to the store to discuss the issue and to resolve it in whatever way might be feasible. So far I haven't had to do that!

I like to learn by trial-and-error. What happens when I do this? When I turn that dial? Is it more effective to use this particular setting or that one? I can easily spend an hour or more "working" the subject, trying different settings, different perspectives, etc. I also usually look online for information about a particular camera body or lens both before and after purchase. That can be useful, especially if I have a question or concern about something. Some people like to go to the store and have the sales personnel set things up for them. When I was buying my gear there was a man in there with his apparently brand-new camera and he was befuddled by some of the settings (I think it may have been his first digital camera), so one of the salespeople was patiently guiding him through the process, showing him or actually doing it for him. That would not work for me!

I'm not a fan of the pixel-peeping shots-of-a-brick-wall kind of scenario. I use the camera and lenses in the way I normally would, shoot the kinds of things I like to shoot and that seems to work just fine for me.

 

[sean000]

I like to do some test shots from a tripod at various apertures, base ISO, with antishock and remote or timed release. Both manual and AF to compare. That way I get a baseline for what I can expect from the lens. I might use a brick wall something with some text on it... or a variety to help me get a feel for sharpness/contrast as well as consistency across the frame and distortion.

With that baseline I will just use the lens while also trying some handheld test shots. If I get results that don't look so good, at least I know it's probably not the lens and can figure out if it's a focus issue, shutter speed or handholding technique issue, etc.

I've read a lot of positive things about the 100-300mm. Enjoy shooting with it!

 

[kallisti]

I usually start off with some handheld test shots in good light with the lens wide open. Will pick a subject at the near focus distance and also at infinity. This gives me a rough impression of whether I got a good sample or a bad sample.

Depending on the day of the week when the new gear arrives, I'll either take it for a spin if I'm going to be out and about with my family or do a more controlled evaluation on a tripod.

Similar to @sean000, I usually shoot an aperture series of something flat with detail/texture/lines that fills the frame to assess for distortion and edge-to-edge sharpness within a week of getting a new lens. Both to see how the corners perform related to sharpness and also to see if there is any skew in the lens (i.e. is the performance uniform across the frame (even if the corners aren't as sharp as the center). Is either the right or left side of the frame not as sharp as the opposite side which can indicate skew (assuming you were shooting dead on to the subject)? Some lenses aren't sharp into the corners/edges (which is a lens design issue). If the right and left sides aren't symmetrical regarding sharpness then it's either a problem with your shooting setup or it's a problem with your camera and/or lens combo (i.e. skew--the lens isn't sitting perfectly perpendicular to the sensor).

One other thing to watch for relates to AF and DOF. Where is your camera/lens combo placing focus within the DOF when using AF? This ends up being a very complex topic that relates to AF on the camera body and also to how well the lens itself behaves regarding AF (could be either a lens design issue or a sample issue).

Say you have a 2 inch DOF for your aperture, focal length, and subject distance. Say you are shooting a person or animal where you want to get the eyes in perfect focus. The final pic won't be the same depending on where your AF places focus with a specific lens/body combo. It is possible that the AF will place focus on the "front end" of the DOF with the eyes being in focus and then extend 2 inches rearward leaving the nose out-of-focus. It's also possible that the AF will place focus on the "rear end" of the DOF with the eyes being in focus and then extend 2 inches frontwards getting the nose in focus but blurring the ears. In either case, the AF got it "right" by putting the eyes in focus, but the resulting images won't be the same. The assumption is that AF will place focus in the middle of the DOF or maybe 1/3 of the way in. This doesn't always happen however. Ideally you want *consistent* results.

This isn't just academic--my Sigma 40mm f/1.4 tends to focus on the front end of the DOF on my D850 which translates into effective back focus as the DOF using AF never extends as far forward as the math would lead one to expect. Worse, this behavior isn't consistent across various subject distances, so I had to come up with a compromise focus adjustment setting for the lens on my D850 that works "most" of the time. It does work as expected on my Z7.

Lens evaluation can be extremely complicated. It's easy to assume that when you get poor AF results that it's a "you" problem. Sometimes that is actually the case. But don't dismiss the possibility that it's actually a problem related to your particular lens sample. Always good to put any new lens through a controlled evaluation relatively soon after you get it so you are within the return window. Or testing a new body against lenses that you know are good. Cameras and lenses are expensive. Make sure they are working optimally when you get them. Nothing worse than getting poor results that are ultimately a gear sample problem and not really a "you" problem. But to determine that, you need to have a good understanding of what the expected behavior should be and then perform controlled experiments to confirm that your gear is performing as expected.

 

[Darmok N Jalad]

I've read a lot of positive things about the 100-300mm. Enjoy shooting with it!

So far I like it quite a bit. I had the O75-300ii, but I wanted to give this P100-300 a try before the return window expired on the Oly. I really notice the extra light, and the dual IS is a big deal for such a long lens. I can drop to 1s shutter at 300mm handheld and still get something useable. Definitely not something I’d try to do, but it’s still amazing to get that kind of result.

 

[Clix Pix]

At this point I have to admit that I have not had all that much experience with Sony's approach to eye-focus, and to date most of it has been with snagging animal eye-focus as opposed to human eye-focus, but I've been fairly pleased with what I've managed to capture so far.....none of it requiring any special effort on my part, which is really rather cool. I am still working my way through learning some of the ins-and-outs of shooting with one particular (long) lens (200-600mm) and have yet to get myself to the store to buy the Wimberley gimbal that I need to put on my tripod for easier management of those long-range shots with that lens.... In general, though, as far as as I can tell, I've lucked out with good lenses doing just exactly what they are meant to do, and that is all that matters as far as I'm concerned.