macOS Abstraction versus conceptual simplicity

[Cromulent]

I've been thinking about this for awhile (and in fact wrote a couple of blog posts about it) but I wanted to get some feedback from you guys.

There seem to be two opposing camps in the programming world. The simplicity at all costs approach and the abstraction is best approach.

C is a good example of simplicity. The language is extremely terse and if you can already program it is relatively straightforward to learn. Once you have mastered the language there really is not all that much more to learn. The disadvantage with this approach is that it requires the programmer to spend inordinate amounts of time implementing all the small little details in their program. There is no standard read line function for instance. This makes it extremely inefficient for writing small applications as you end up spending several orders of magnitudes longer implementing simple functionality yourself. Of course you end up building a library yourself over time and just reusing that in all your applications which eventually saves you time.

Then there is the abstraction camp. Java is a great example of this. It has extremely comprehensive libraries and frameworks and attempts to get rid of the problem that C programmers face by abstracting away some of that complexity to the standard library and by providing an object orientated programming idiom which hides some of the implementation detail. This means that for small applications Java is much better than C as you can just use the wealth of pre-written standard library functions to get the job done for you.

The problem with the abstraction method is that it removes the one thing that all programmers have in common. An interest in computers and by inference computer architecture. My contention is that by hiding the implementation behind an object orientated idiom you are turning something that programmers naturally find easier to understand into something which is outside of their realm of experience.

With C and other bare bones languages the time spent to implement a program is probably going to be equal to the amount of time a Java programmer takes getting used to using the absolutely huge Java frameworks. Have you ever tried to use Java EE? It is a mind boggling task which is just made harder by the attempts to hide the implementation behind abstractions. Programmers no longer have the option to really understand what they are doing, instead they have to rely on understanding abstractions which are not necessarily any easier to understand and are often poorly explained in documentation. They also must grapple with 1,000 page documents which attempt to explain things in a friendly fashion.

By the time you have read most of the documentation of these frameworks one can not help but wonder what you could have achieved in the mean time if you had just put your head down and got on with it in C.

Ideas, comments, opinions?

 

[lee1210]

Obviously higher-level languages are very popular, and it's unlikely that it's because people that use these languages don't know that C or other nearer-machine languages are available.

My opinion is that as machines get increasingly more powerful the actual speed difference for most programs when using C vs. Java, for example, is going to be very small. Certainly keeping performance in mind is important, but often speed problems are due to algorithmic complexity, or other issues, not the language a program is implemented in. However, if you can program more safely, more quickly, using more comprehensive libraries without severely affecting performance you will often end up on top.

I can speak to the differences pretty definitively, because my job requires me to use fortran, C, Java, Perl, Javascript, shell, etc. If you only had expert C programmers, for example, touch your C code, you are probably unlikely to see the most common problems like memory management, overflows, mismatched parameters, etc. Over the last 25 years, I can assure you the people working on this code base have not all been experts. I have seen tons of these sorts of errors in the C code. I'm not talking about logical errors, i'm not talking about errors implementing an algorithm, etc. I'm just talking about janitorial issues. In Java, these issues can occur as well. You can still leak references, hence leaking memory. If you use arrays instead of collections, etc. you can definitely still overflow, etc. Null pointer exceptions are still there, etc. But there are fewer janitorial issues, and a lot of convenient features that make writing code faster. Not just libraries, but things like for-each, garbage collection, synchronization intrinsics, etc.

While it may be true that you could write the same program in C in the same time as it might take you to scour the Java documents figuring out the libraries to implement the same thing, this isn't going to be the case every time you write a program. Eventually you'll be familiar with the libraries, so your 1000th Java program will probably be much easier to write than your 1000th C program with the same complexity. The Java version is likely to have fewer bugs related to basic maintenance. Granted, by the time you've gotten to your 1000th C program you may have a library to do a lot of things for you, but if you ever want anyone else to work on your code you have to go to the trouble of completely documenting everything, keeping the docs updated as you make changes, etc. The Java libraries are already documented, even if it isn't always the easiest to find what you need. This means that when your C program uses a function in your great C library, a person reading the code for the first time has zero chance of knowing that that thing does. In Java, a program that uses the libraries extensively will be easier for a new programmer to grasp immediately if they have a familiarity with the Java libraries.

Another big issue is APIs. On Linux, obviously C is king, and you'll need C to interact with the system. However, on OS X, you can use C, but you're going to have to be doing work with Objective-C eventually to interact with the system. Once you're using it for your OS hooks, why not leverage its power in other ways, using NSString, NS(Mutable)Array, etc.? Why not use fast enumeration?

Also, when it comes to studying docs, if you spend time writing your own library in C instead of reading docs for a higher-level languages libraries, you may end up with something great. But unless you can really get others behind it, no one else will be using this library. On subsequent projects you start, you can leverage this code. However, when you get involved with someone else's project, you probably can't. And when you start reading other people's code, knowing all about your great library isn't going to help you. A good familiarity with the docs for a language like Java, however, will apply to nearly any Java you read. This means that reading the docs to get familiar with the libs for your own project isn't a sunk cost.

This post sounds like I'm arguing that higher-level languages like Java are better. That's not really the point, but I know what side of the fence you're on, so I'm trying to present some reasons that higher-level languages are preferred by some. I especially prefer them in collaborative environments, where some team members may be novices or beginners. There are certainly times when a JVM is simply unavailable, performance is so critical that no amount of JIT in the world will garner enough speed, etc.

I certainly think a good foundation in C is vital, but I think a lot of programmers don't actually love computers and computer architecture. You're passionate about programming, computing, etc. I am too. But for some people, programming is a job. They don't want to use a computer for anything but email and surfing the web outside of the hours of 9-5. A lot of times this means they will prefer the abstraction if it makes their job easier.

I don't think any real conclusions are going to be reached here. Everyone is going to have different ideas about this It's best to be comfortable with both, and it's nice to be able to stick them together with JNI if you have to.

-Lee

 

[Cromulent]

Obviously higher-level languages are very popular, and it's unlikely that it's because people that use these languages don't know that C or other nearer-machine languages are available.

My opinion is that as machines get increasingly more powerful the actual speed difference for most programs when using C vs. Java, for example, is going to be very small. Certainly keeping performance in mind is important, but often speed problems are due to algorithmic complexity, or other issues, not the language a program is implemented in. However, if you can program more safely, more quickly, using more comprehensive libraries without severely affecting performance you will often end up on top.

Good point. Certainly I imagine in a commercial environment that speed of implementation is an important factor as well as maintainability. Higher level languages certainly have the edge there as they provide more in their standard libraries. But then you come across languages like C++ which are hard to categorise.

They attempt to offer the object orientated programming idiom to the programmer as a means of easing the reuse of code but they fail to remove the parts of the language which are well known to be confusing. In fact I have never seen the point of C++. It is a language which attempts to make C better but instead of removing C's well known disadvantages such as manual memory management and confusion with pointers it keeps them and then destroys the most important advantage of C code. It is simple, extremely terse and generally fairly easy to learn the basics of. With C++ you have none of that.

I can speak to the differences pretty definitively, because my job requires me to use fortran, C, Java, Perl, Javascript, shell, etc. If you only had expert C programmers, for example, touch your C code, you are probably unlikely to see the most common problems like memory management, overflows, mismatched parameters, etc. Over the last 25 years, I can assure you the people working on this code base have not all been experts. I have seen tons of these sorts of errors in the C code. I'm not talking about logical errors, i'm not talking about errors implementing an algorithm, etc. I'm just talking about janitorial issues. In Java, these issues can occur as well. You can still leak references, hence leaking memory. If you use arrays instead of collections, etc. you can definitely still overflow, etc. Null pointer exceptions are still there, etc. But there are fewer janitorial issues, and a lot of convenient features that make writing code faster. Not just libraries, but things like for-each, garbage collection, synchronization intrinsics, etc.

Don't get me wrong, I like Java. I think it is a very clean language and is (in general) well documented. In fact it is the primary language that my Comp Sci degree uses (which starts worryingly soon) and I'm looking forward to getting a chance to work with it a bit more.

But my problem is not so much the language itself, more the philosophy that the language imparts on framework developers. I've been reading numerous Java blogs and the general attitude seems to be "why do I need to understand that?" rather than "I wonder how that was done?". Maybe it is just me, but the first time I see something new I always like to understand how it works rather than just getting on with it (not a particularly productive style I'll grant you that).

While it may be true that you could write the same program in C in the same time as it might take you to scour the Java documents figuring out the libraries to implement the same thing, this isn't going to be the case every time you write a program. Eventually you'll be familiar with the libraries, so your 1000th Java program will probably be much easier to write than your 1000th C program with the same complexity. The Java version is likely to have fewer bugs related to basic maintenance. Granted, by the time you've gotten to your 1000th C program you may have a library to do a lot of things for you, but if you ever want anyone else to work on your code you have to go to the trouble of completely documenting everything, keeping the docs updated as you make changes, etc. The Java libraries are already documented, even if it isn't always the easiest to find what you need. This means that when your C program uses a function in your great C library, a person reading the code for the first time has zero chance of knowing that that thing does. In Java, a program that uses the libraries extensively will be easier for a new programmer to grasp immediately if they have a familiarity with the Java libraries.

That is another good point. I agree that large standard libraries offer a means for many different programmers to work on a project without having to constantly learn new skills.

Another big issue is APIs. On Linux, obviously C is king, and you'll need C to interact with the system. However, on OS X, you can use C, but you're going to have to be doing work with Objective-C eventually to interact with the system. Once you're using it for your OS hooks, why not leverage its power in other ways, using NSString, NS(Mutable)Array, etc.? Why not use fast enumeration?

I was hoping to avoid Objective-C as I think it is actually the perfect mix. It offers higher level functions for general run of the mill development but if you want to drop down into standard C you are perfectly capable of doing so. Plus even in the higher level functions you still have access to standard C.

One of the biggest shames I think is that C++ became more popular than Objective-C.

Also, when it comes to studying docs, if you spend time writing your own library in C instead of reading docs for a higher-level languages libraries, you may end up with something great. But unless you can really get others behind it, no one else will be using this library. On subsequent projects you start, you can leverage this code. However, when you get involved with someone else's project, you probably can't. And when you start reading other people's code, knowing all about your great library isn't going to help you. A good familiarity with the docs for a language like Java, however, will apply to nearly any Java you read. This means that reading the docs to get familiar with the libs for your own project isn't a sunk cost.

True again. I guess it just depends where you want to spend the time. Ultimately the time spent learning the Java frameworks will pay off in the end but for one man developers learning Java EE in particular is a mammoth task.

This post sounds like I'm arguing that higher-level languages like Java are better. That's not really the point, but I know what side of the fence you're on, so I'm trying to present some reasons that higher-level languages are preferred by some. I especially prefer them in collaborative environments, where some team members may be novices or beginners. There are certainly times when a JVM is simply unavailable, performance is so critical that no amount of JIT in the world will garner enough speed, etc.

I certainly think a good foundation in C is vital, but I think a lot of programmers don't actually love computers and computer architecture. You're passionate about programming, computing, etc. I am too. But for some people, programming is a job. They don't want to use a computer for anything but email and surfing the web outside of the hours of 9-5. A lot of times this means they will prefer the abstraction if it makes their job easier.

I don't think any real conclusions are going to be reached here. Everyone is going to have different ideas about this It's best to be comfortable with both, and it's nice to be able to stick them together with JNI if you have to.

-Lee

Thanks for responding. It is certainly interesting to hear other takes on the subject and that definitely helps with my learning about programming in general.

 

[Sander]

Maybe it is just me, but the first time I see something new I always like to understand how it works rather than just getting on with it (not a particularly productive style I'll grant you that).

I disagree. I think this is a highly productive style in the long run.
The problem with abstractions is that it is very hard to make good ones, meaning they aren't "leaky". Check Joel On Software's excellent article on this subject. When you don't understand how something works, you'll be at a loss when your abstractions leak. When that happens, you will either have to invest time understanding it (which comes at a highly inconvenient time, because you found out it was leaking when your project got under pressure), or you'll have to walk, tail between your legs, to the bearded, grumpy guru at the end of the hallway who understands the abstraction and probably advised you to be careful with it in the first place (when all the others were jumping up and down with excitement over this new thing which would make software development an order of magnitude more efficient). So: Keep this attitude!

In fact I have never seen the point of C++. It is a language which attempts to make C better but instead of removing C's well known disadvantages such as manual memory management and confusion with pointers it keeps them and then destroys the most important advantage of C code. It is simple, extremely terse and generally fairly easy to learn the basics of. With C++ you have none of that.

I happen to like C++ a lot. Most of its ugly parts come from its compatibility with C, but it offers additional stuff. I haven't done manual memory management (or any resource, really) in C++ since I've seen the light of RAII. In fact, I'd say RAII causes much less problems than GC does.

The problem is that many programmers approach C++ as being C "with some extra stuff", replace "malloc" with "new" and "free" with "delete", and call the resulting code C++.

But to answer more directly to your original posting; I don't think "conceptual simplicity" necessarily stands in the way of "abstraction". A good abstraction is conceptually simple. That just means good abstractions are really hard. Look at the Boost libraries for instance. Often, you'd think "Why did it take so long to make a library for XYZ, it should be really simple, and why is there so much code to it?" and then you'd start reading the rationales and realize how much stuff needed to be solved which you didn't even think about. I think many other frameworks/languages (C#, I'm looking at you) take the approach of "Ah come on, it works in most of the cases".

 

[savar]

I didn't have time to read through the whole thread, yet, but I think you might find this article very relevant to your intellectual pursuit:

http://www.joelonsoftware.com/articles/LeakyAbstractions.html

The long and short of it: abstractions are a useful tool, but taken to the extreme would result in total absurdity. We need to be careful about encapsulating dangerous behavior behind an abstraction. That seems to be what you're referring to with your Java example.

Edit: Just read the thread. Dang, can't believe Sander posted that link before me! Anyway, good to see others here like to talk about these kinds of issues. In my mind, this stuff is still largely undecided, which is a shame considering how much of the world still needs to be automated.

 

[Sydde]

The nice thing about Objective-C and Cocoa is that the full range of tools is accessible to the programmer. You can write an entire, useful application without straying very far from the metaphor or you can go as deep as you want/need into the system (I currently face the daunting task of learning how to communicate with Mach without blowing things apart). From what I can see, Objective-C cleanly expands the conciseness of C into something that is very easy to work with. (I can handle NSString objects with ease, without, I confess, having any clue about the difference between Unicode and UTF-8, just that it sometimes matters.)

 

[Arran]

I don't think it's a battle between abstraction vs conceptual simplicity. To me, conceptual simplicity is the goal that's attained through abstraction.

Of course, it all hinges on how you define "simple". Consider a trip to a grocery store. Is it simpler to grab flour, butter salt and apples from well-stocked aisles? Or is it more "conceptually simpler" to: mill your home-grown wheat into flour, keep a cow you milk for home-made butter, evaporate seawater to crystallize salt and, lastly, tend to a backyard orchard?

Don't get me wrong, focusing on all the lower-level details can make for a very fulfilling and rewarding life - but you might not have time to make an apple pie, never mind eat it. I guess the trick is to work at the level of abstraction that makes you happy and gives you time to do the things that interest you.

Taking the analogy to an extreme level, I believe it was Carl Sagan who said, "If you wish to make an apple pie from scratch, you must first invent the universe."

...and that certainly wouldn't leave time for anything else

 

[Cromulent]

I disagree. I think this is a highly productive style in the long run.
The problem with abstractions is that it is very hard to make good ones, meaning they aren't "leaky". Check Joel On Software's excellent article on this subject. When you don't understand how something works, you'll be at a loss when your abstractions leak. When that happens, you will either have to invest time understanding it (which comes at a highly inconvenient time, because you found out it was leaking when your project got under pressure), or you'll have to walk, tail between your legs, to the bearded, grumpy guru at the end of the hallway who understands the abstraction and probably advised you to be careful with it in the first place (when all the others were jumping up and down with excitement over this new thing which would make software development an order of magnitude more efficient). So: Keep this attitude!

Thanks for posting that. It was an interesting read.

I happen to like C++ a lot. Most of its ugly parts come from its compatibility with C, but it offers additional stuff. I haven't done manual memory management (or any resource, really) in C++ since I've seen the light of RAII. In fact, I'd say RAII causes much less problems than GC does.

The problem is that many programmers approach C++ as being C "with some extra stuff", replace "malloc" with "new" and "free" with "delete", and call the resulting code C++.

But to answer more directly to your original posting; I don't think "conceptual simplicity" necessarily stands in the way of "abstraction". A good abstraction is conceptually simple. That just means good abstractions are really hard. Look at the Boost libraries for instance. Often, you'd think "Why did it take so long to make a library for XYZ, it should be really simple, and why is there so much code to it?" and then you'd start reading the rationales and realize how much stuff needed to be solved which you didn't even think about. I think many other frameworks/languages (C#, I'm looking at you) take the approach of "Ah come on, it works in most of the cases".

Good C++ code I whole heartedly admit is just as good as any other language but the problem is that (comparatively speaking) very few people write good C++ code. I know I would be unable to as I would avoid the STL like the plague and attempt to use C++ as a kind of faster version of Java.

The problem is that C++ offers so many things that it starts to become counter productive. Writing C++ code and writing good C++ code are miles apart, and there are so many different ways a programmer can make mistakes in C++ when compared with C that it becomes more like an art form rather than an exercise in reason.