macOS pointers - please explain

[Fukui]

the language i'm learning is C, but my main aim is to move on from C to ObjC/Cocoa

Yea, thats one of the thing I like about Obj-C, you learn C and thats a strength, you also learn OO principles along with it. That and I love the syntax and API, it somehow doesn't feel so much like programming, just looking at plain C or Java or J-Script, I find it feels too terse and "machine" like. Though C et all is very powerful.

 

[mrichmon]

Right, but I wonder there couldn't be a kind of compromise, instead of "Must Garbage Collect" or "Must Hide Pointers" is to provide a layered approach, like a base implemenation using functions, pointers, and no collection or bounds checking, then based on that build on an object layer, then add collection etc, then there wouldn't be any translation "layers" like JNI or the C# bridge...


I didn't know garbage collectors manipulate pointers... I though they just keep references to the memory (pointer) and the null it once it had no references... but then again I guess thats why they hide the pointers, thats how they count references! I thought instead the runtime would check if the code had pointers to the location of an object or struct, then if all the pointers were nulled to that particular location, then it would be freed...

Its interesting, I wonder how they'll implement the garbage collector in Obj-C, probablly only objects could be collected, but then they don't hide the pointers...

Thanks for the info.

The issue is that good garbage collectors move the object around in memory. What you need is for the reference to be independent of the location that the object is stored in memory. A pointer references a particular location in memory so if the object is moved then the pointer no longer points to the object.

It's not clear what precisely you mean by a layered approach. But in general you can do whatever you want when designing a programming language. The trick is working out how to implement it efficiently.

The GC used in Objective-C is a rather simple collector and has several serious flaws. The most obvious of which is that it relies on reference counting. With a reference counting collector, any loop of objects such as object A holding a reference to object B and object B holding a reference to object A will never be collected no matter whether no other objects hold a reference to A or B. This results in a memory leak.

 

[mj_1903]

The GC used in Objective-C is a rather simple collector and has several serious flaws. The most obvious of which is that it relies on reference counting. With a reference counting collector, any loop of objects such as object A holding a reference to object B and object B holding a reference to object A will never be collected no matter whether no other objects hold a reference to A or B. This results in a memory leak.

If you end up with a situation like that then it is time to either rewrite that block of code or move to autoreleased objects.

I would hate to think of an instance where two objects would retain references of each other... you are more likely going to end up with a crash than a memory leak.

 

[Fukui]

It's not clear what precisely you mean by a layered approach. But in general you can do whatever you want when designing a programming language. The trick is working out how to implement it efficiently.

Well, like in Java, everything is hidden, and now they're complicating the langiuage even more with 1.5, C# is adding lots of complicated syntax in 2.0 and 3.0, its just seems like they firstly try to eliminate alot of complexity, then later, they realize, oops we need to do XYZ but we can't becuase the base is too inflexible, so we gotta extend the language.

Why cant a language instead build on a more flexible (but not as safe) base (like C) then using C (the base implementation) add abstractions that make it simpler as one goes up along the levels. IOW, say I had Obj-C with a full garbage collector and a JIT and bounds checking etc, I could call code from C, or assembly or procedural code when I want, but no native interface non-sense etc, because its actually just all C code I'm using, the pre-processor, runtime handles it all for me..

JMHO but its seems kind of backwards to provide a simple and inflexible base then add complexity to it later (like Java and C#)...I guess I just "want it all"

 

[mrichmon]

If you end up with a situation like that then it is time to either rewrite that block of code or move to autoreleased objects.

I would hate to think of an instance where two objects would retain references of each other... you are more likely going to end up with a crash than a memory leak.

That is far from true. A double linked list is a commonly used data structure and requires exactly that sort of structure. If you then release the head of the list the elements in the list will not be collected by a garbage collector that is based on reference counting. Instead you need at least a mark and sweep collector, or a generational collector.

How exactly do you think things will crash? By magic?

 

[mrichmon]

Well, like in Java, everything is hidden, and now they're complicating the langiuage even more with 1.5, C# is adding lots of complicated syntax in 2.0 and 3.0, its just seems like they firstly try to eliminate alot of complexity, then later, they realize, oops we need to do XYZ but we can't becuase the base is too inflexible, so we gotta extend the language.

The syntax of a language has very little relation with the semantics provided by the language. The trick with extending a language like Java is to do so in a way that will work with previously written code since there is a huge user base already.

Why cant a language instead build on a more flexible (but not as safe) base (like C) then using C (the base implementation) add abstractions that make it simpler as one goes up along the levels. IOW, say I had Obj-C with a full garbage collector and a JIT and bounds checking etc, I could call code from C, or assembly or procedural code when I want, but no native interface non-sense etc, because its actually just all C code I'm using, the pre-processor, runtime handles it all for me..

You could. Whether you would get the performance you want is an open question. Then there is the question of whether anyone else would want to use the language. There are literally thousands of langauges out there implemented by someone who wanted a programming language to work in a different way.

Also, getting Objective-C to have a full garbage collector is a major technical challange. Working out the appropriate semantics and behavior for a JIT for Objective-C is also a major technical challenge, particularly since Objective-C is not an interpreted language. If you are running C code then there is no runtime to be concerned with.

All of these things you are suggesting are major pieces of work, each of which a good grad student would need to study for 3 years or so. If they actually work out an implementation then the student would have done sufficient work to get a PhD.

But you are right, if you had Objective-C with a good GC. And if you redesigned some of the ugly semantics out of Objective-C. Then it is likely that you would be able to call native code directly. However at this point you would not have a langauge that produces binary code that is portable across any number of platforms like Java and C# is portable.

JMHO but its seems kind of backwards to provide a simple and inflexible base then add complexity to it later (like Java and C#)...I guess I just "want it all"

That's not quite how it works but based on this short discussion I can see why you view it this way. In my previous posts I have given a highly simplified the explaination of certain language elements. In practice you cannot view these elements in isolation since many different aspects of a language interact and force certain other choices to be made and similarly prevent you from using certain techniques.

Ultimately langauge design is a complex field.

 

[Fukui]

You could. Whether you would get the performance you want is an open question. Then there is the question of whether anyone else would want to use the language. There are literally thousands of langauges out there implemented by someone who wanted a programming language to work in a different way.

I guess the idea would be that since its natively C based, it would be easy to integrate with existing code bases... programers could choose the level abstraction they wanted to code in, and since the syntax and base API are the same, then a lot of code can be copied and pasted from eachother... (though I guess with a good parser any java code could and classes could be copied into C# form pretty easily since they seem so similar...)

I remember once bill gates said "the difference between languages is largely just syntax," which was one justification for the CLR... its an interesting idea I wonder if it has some truth to it? Kind of sucks in a way if true, it makes it sound as if there's no difference anymore or forever....

Thanks for your detailed reply BTW.

 

[HiRez]

The GC used in Objective-C is a rather simple collector and has several serious flaws. The most obvious of which is that it relies on reference counting. With a reference counting collector, any loop of objects such as object A holding a reference to object B and object B holding a reference to object A will never be collected no matter whether no other objects hold a reference to A or B. This results in a memory leak.

First of all, what Obj-C garbage collector are you talking about? Is it for some open-source implementation or something...surely Apple is not adding a GC to theirs are they? Anyway, couldn't you just make it so that if the GC detects such a reference loop while trying to delete an object the user has released, it nulls out the references on the other side of the object as well (IOW, the objects connected to that object with two-way pointers)? In the case of a double-linked list you'd still have to do some manual patching of references when deleting elements, but I would expect to have to do that myself anyway.

 

[Fukui]

First of all, what Obj-C garbage collector are you talking about? Is it for some open-source implementation or something...surely Apple is not adding a GC to theirs are they?

Yep, check

- (void)finalize

This is not yet implemented in Tiger, but its comming.

 

[HiRez]

- (void)finalize

This is not yet implemented in Tiger, but its comming.

Wow. How did I miss that!? Intriguing, yet scary!

 

[GeeYouEye]

That is far from true. A double linked list is a commonly used data structure and requires exactly that sort of structure. If you then release the head of the list the elements in the list will not be collected by a garbage collector that is based on reference counting. Instead you need at least a mark and sweep collector, or a generational collector.

How exactly do you think things will crash? By magic?

But then you never see "NSDoublyLinkedList" in any framework; the right tools for the right job, structs in this case.

And the currently implemented reference counting scheme is not a garbage collector. The runtime calls -dealloc on any object with a retain count of 0. But that's it. -dealloc can be implemented however you want it to be. -dealloc doesn't have to deallocate the object. It could paint a pretty picture on the screen, and NSLog "screw you, I'm not going to be deallocated" to the console. The only guarantee of anything being actually deallocated is the [super dealloc] message that's almost always at the end of the method. But it doesn't have to be.

The ability to override dealloc is definitely a good thing, especially if you're working with expensive singletons (anyone on the Cocoa-dev list? there was a HUGE thread about this recently).

Then there's the whole autorelease pool thing... Personally, I find that has just as much flexibility in memory management as a GC, if not more.

A garbage collector would be very difficult to implement in Objective-C, because of the language's dependence on C. and it's impure Object-Orientation (how many classes store numbers as ints and floats rather than NSNumbers?). Objective-C has a lot of power because one can bridge the gap between interface (OO) and implementation (procedural C) without any special constructs, but this means there's a lot of pitfalls. I imagine either the Obj-C GC will either only work on objects, or will be quite an accomplishment, a universal C garbage collector. Given the presence of -finalize, I'm going to guess the former.

 

[devman]

C-compatible garbage collector info. http://www.iecc.com/gclist/GC-faq.html Great Circle by geodesic was one that I knew of for C++. You can still find reviews of the product if you google it. Here's one

Great Circle 2.1

Until recently, Great Circle was positioned as a garbage-collection utility for C and C++ programmers. It still does that, but Geodesic Systems has found that while most C programmers understand that they need to fix memory leaks (which Great Circle does), they may not understand how garbage collection can help.

Consider a C program with dynamic memory allocations-malloc calls to allocate memory, and free calls to release that memory. Normally if you malloc without a later free, the memory leaks. If you free a block twice, the memory heap becomes corrupt. Once you link it into your program, Great Circle lets you dispense with free calls. Instead, it scavenges memory in the background and periodically releases dead memory blocks-blocks for which no valid pointer is in scope. Great Circle can also report what it's doing.

Effectively, Great Circle acts as both a diagnostic and an immediate cure for memory leaks. Since it can fix memory leaks coming from third-party libraries and DLLs, as well as leaks coming from your own code, it can sometimes prove indispensable. And if you rely on Great Circle from the beginning of a project, instead of constantly trying to match your memory allocations, you can save an enormous amount of programming time.

Great Circle doesn't really compete directly with BoundsChecker-it's more of a complementary tool. It detects only a few out of the thousands of possible problems in Windows programs, while BoundsChecker detects essentially all of them. On the other hand, BoundsChecker does not actually fix anything, while Great Circle almost magically turns a program that leaks like a sieve into something solid and stable without requiring you to change a line of code.

All just FYI. (note: geodesic are no longer around. it was about 6-7 years ago I saw a team using great circle)

 

[Fukui]

Wow. How did I miss that!? Intriguing, yet scary!

Yea, and I'm wondering why we need to implement finalize if a garbage collector is gonna take care of dealoc-ing everything anyways... which leads me to think its just collecting objects, structs and c arrays etc are gonna have to be freed manually... just guessing.

 

[HiRez]

which leads me to think its just collecting objects, structs and c arrays etc are gonna have to be freed manually... just guessing.

Probably so, but that would be ok with me, I rarely find myself actually using C arrays or structs when I can use NSArray or a custom NSObject subclass...IMO better to keep everything in "Objectville" unless there's a compelling reason not to. I do tend to use a lot of floats and ints instead of NSNumbers, however since those don't need to be allocated manually, I don't know why it's be a problem to have them garbage-collected. I wonder if Apple could just automatically store a programmer's primitive scalar types as NSNumber objects behind the scenes anyway, thus allowing them to be directly stored into NS collection classes. I rarely use structs except for NSPoint, NSRange, NSSize, etc, which are structs but could, and I think should, just be implemented as first-class objects anyway.

 

[HiRez]

Yea, and I'm wondering why we need to implement finalize if a garbage collector is gonna take care of dealoc-ing everything anyways...

I think -finalize: gives you a chance to do your own clean-up before your object is trashed, for example to close a file or net connection that your object opened, or to remove the object from an NSNotificationCenter. IOW, all the same stuff you would do in -dealloc: except that you wouldn't necessarily have to null out references that your object alone creates and accesses (although it's probably a good idea to do so anyway. I remember when I was using Java, I got into the habit of nulling out references I was done using manually, even though it had automatic garbage collection. This ensured the object would be collected ASAP and didn't leave a dangling pointer somewhere.

 

[Catfish_Man]

I think -finalize: gives you a chance to do your own clean-up before your object is trashed, for example to close a file or net connection that your object opened, or to remove the object from an NSNotificationCenter. IOW, all the same stuff you would do in -dealloc: except that you wouldn't necessarily have to null out references that your object alone creates and accesses (although it's probably a good idea to do so anyway. I remember when I was using Java, I got into the habit of nulling out references I was done using manually, even though it had automatic garbage collection. This ensured the object would be collected ASAP and didn't leave a dangling pointer somewhere.

I'm a little unclear on why the gc can't simply call -dealloc... release/retain/autorelease will be overridden to be empty methods when gc is turned on, so if it weren't for the dealloc/finalize difference, than it seems like you could simple enable gc on an existing program and have it work. I'd be interested in hearing the logic behind this decision.

 

[mrichmon]

I remember when I was using Java, I got into the habit of nulling out references I was done using manually, even though it had automatic garbage collection. This ensured the object would be collected ASAP and didn't leave a dangling pointer somewhere.

By definition it is not possible to leave a dangling pointer in Java. This is because Java does not use pointers, rather Java uses "references". A internally a reference consists of more than just the memory location for an object.

The way you wind up with a dangling pointer is by deallocating the memory without voiding the pointer to that memory. In Java the programmer cannot explicitly deallocate memory so it is impossible for a reference to exist without the object the reference refers to also existing.

 

[Fukui]

I think -finalize: gives you a chance to do your own clean-up before your object is trashed, for example to close a file or net connection that your object opened, or to remove the object from an NSNotificationCenter.

That makes more sense, yea, it'll be interesting to see what they do.
I hope they allow us to still controll the collection or turn it off for some objects like [object garbageCollect:NO]. I'll be sooooooo happy if/when the garbage collector nils out invalid pointers... no more messages to "dead objects" and crashes. I can just check "if (someobject)" and always expect if there's an object gone, I can re-instantiate it, or catch a bug without crashing most of the time.

 

[mrichmon]

I'm a little unclear on why the gc can't simply call -dealloc... release/retain/autorelease will be overridden to be empty methods when gc is turned on, so if it weren't for the dealloc/finalize difference, than it seems like you could simple enable gc on an existing program and have it work. I'd be interested in hearing the logic behind this decision.

The way a GC would deallocate the memory is simply by calling dealloc() after ensuring any release/finalize methods are invoked. The problem is how to implement a GC in the context of a language that uses pointers.

Efficient garbage collectors move objects around in memory such that live objects end up in one part of memory, garbage ends up in another part of memory. The garbage area is deallocated in one block. In a langauge that uses pointers, moving objects means that you need to find all the pointers to a given object then move the object, then update the pointers. Then repeat this for each object you are interested in. This is an expensive proposition. It is also difficult to identify values that are used as pointer addresses since any integer type can be used to store a memory address. (ie long foo = *some_pointer; another_pointer = foo).

References remove this problem since it is generally not possible to assign an address to a reference in order to access a different object.

 

[Catfish_Man]

<snipped>

Didn't really answer my question. Garbage collectors for C based languages are nothing new, I was just wondering about why they had to make a new objective-c method that does exactly the same thing as -(void)dealloc (specifically, release any resources owned by the object).

 

[devman]

Didn't really answer my question. Garbage collectors for C based languages are nothing new, I was just wondering about why they had to make a new objective-c method that does exactly the same thing as -(void)dealloc (specifically, release any resources owned by the object).

speculation_mode=on.

Because if they are adding GC then people won't be calling dealloc. In GC environments finalize is called by the GC when there are no more references to the object. Thus it is the hook you need to release any resources you may be holding, or to undo any external state stuff you may have done during construction or during the object's life.

So, it could be prepping for GC.

speculation_mode=off.

 

[Catfish_Man]

speculation_mode=on.

Because if they are adding GC then people won't be calling dealloc. In GC environments finalize is called by the GC when there are no more references to the object. Thus it is the hook you need to release any resources you may be holding, or to undo any external state stuff you may have done during construction or during the object's life.

So, it could be prepping for GC.

speculation_mode=off.

Well, it explicitly says in the documentation that it's called by the GC. However, dealloc is never called by programmers anyway, it's called by the retain counting system, which can be thought of as a primitive form of GC.

 

[devman]

Well, it explicitly says in the documentation that it's called by the GC. However, dealloc is never called by programmers anyway, it's called by the retain counting system, which can be thought of as a primitive form of GC.

Ok, excluding your definition of GC (which includes the current retain system):

If they're adding what most people consider GC to be, then retain will become a no-op.

 

[Catfish_Man]

Ok, excluding your definition of GC (which includes the current retain system):

If they're adding what most people consider GC to be, then retain will become a no-op.

release/retain/autorelease will be overridden to be empty methods when gc is turned on

I know this. I know (roughly speaking) how it will work, I've read the emails to the gcc list, and the gnustep list, and the compiler documentation. I've looked into creating GC'd languages, and read about many others. My question is not about gc. It's about why they made a new name for dealloc instead of using the old one.

 

[devman]

I know this. I know (roughly speaking) how it will work, I've read the emails to the gcc list, and the gnustep list, and the compiler documentation. I've looked into creating GC'd languages, and read about many others. My question is not about gc. It's about why they made a new name for dealloc instead of using the old one.

oh, ok. Well we have to speculate again. I can think of two reasons.

If you were starting fresh and it was to be a GC environment what would you call the plugin point for people to "do whatever they have to do" before an object is GCed? dealloc is hardly a good name for this.

Also, Java is a huge influence here. It's Java that put GC environments back into the mainstream.