macOS NSSream, frustrating the hell out me!!

[turner296]

Application Outline
I'm trying to build an application that backs up a folder and uploads it to a server, on online backup storage.

The Problem (or at least I think it is...)
I have the HTTP request within a NSData object, and I'm writing it to an NSOutputStream but when its finished writing and the NSInputStream comes into play I get the wrong response from the server. I thought it must be my request data within the NSData object but then I write that to file. I then compared it to a 1,000,000% (must state that this request is DEFINITELY working) working request that I had written to file using a PERL script, compared the two requests and they were IDENTICAL in every way. I even tried the request I outputted from COCOA into the PERL script and it worked perfectly, so I know the problem lies with the coding of my NSOutputStream.

I logged all the data being sent at each buffer when writing to the NSOutputStream and it does go through right till the end of data till there's no more bytes left, so its got to be sending all of it. It just seems to be the response from the server.

Solution???
I dont want to use the NSURL option because it doesn't really allow a delegate to track the progress of the upload, it basically says connection open, recieved and closed. I need help, considering this is main function of my application I can't go any further with my design or coding of my project till this is resolved.

Code

- (void)start {
	
	NSURL * uploadURL;
	NSHost * uploadHost;
	
	uploadURL = [NSURL URLWithString:uploadServer];
	uploadHost = [NSHost hostWithName:[uploadURL host]];
	
	[NSStream getStreamsToHost:uploadHost
						  port:80
				   inputStream:&inputStream
				  outputStream:&outputStream];
	
	[inputStream retain];
	[outputStream retain];
	
	[inputStream setDelegate:self];
	[outputStream setDelegate:self];
	
	[inputStream scheduleInRunLoop:[NSRunLoop currentRunLoop]
						   forMode:NSDefaultRunLoopMode];
	[outputStream scheduleInRunLoop:[NSRunLoop currentRunLoop]
							forMode:NSDefaultRunLoopMode];

	
	[inputStream open];
	[outputStream open];
	
}

- (void)stream:(NSStream *)stream handleEvent:(NSStreamEvent)eventCode {
	
    //NSLog(@"stream:handleEvent: is invoked...");
	
	if (stream == outputStream) {
		
		//NSLog(@"output stream: active!");
		
		switch (eventCode) {
			case NSStreamEventHasSpaceAvailable:
			{
				
				// get the range of data to send
				// put it into the buffer
				// send it
				// increase the next range
				
				unsigned int data_length = [uploadData length];
				unsigned int length      = ((data_length - byteIndex >= 1024) ? 1024 : (data_length-byteIndex));
				NSRange dataRange        = NSMakeRange(byteIndex, length);
				uint8_t buffer[length];
				
				[uploadData getBytes:buffer
							   range:dataRange];
				
				[uploadedData appendBytes:buffer
								   length:length];
				
				length = (NSInteger)[(NSOutputStream *)stream write:(const uint8_t *)buffer
														  maxLength:(NSUInteger)length];
				
				byteIndex += length;
				
				NSLog(@"Sent %i of %i.", byteIndex, data_length);
				
				break;
			}
			case NSStreamEventErrorOccurred:
			{
				
				NSLog(@"output stream: (ERROR) %@", [[stream streamError] localizedDescription]);
				
				[stream close];
				[stream release];
				
				stream = nil;
				
				break;
			}
			case NSStreamEventEndEncountered:
			case NSStreamEventOpenCompleted:
			default:
				break;
		}
	}
	
	else if ( stream == inputStream ) {
		
		NSLog(@"input stream: active!");
		
		switch (eventCode) {
			case NSStreamEventHasBytesAvailable:
			{
				if( ! responseData ) {
					responseData = [[NSMutableData data] retain];
				}
				
				uint8_t buf[1024];
				unsigned int len = 0;
				
				len = [(NSInputStream *)stream read:buf maxLength:1024];
				
				if (len) {
					[responseData appendBytes:(const void *)buf
									   length:len];
					// bytesRead is an instance variable of type NSNumber.
					[bytesDownloaded setIntValue:[bytesDownloaded intValue]+len];
				} else {
					NSLog(@"no buffer!");
				}
				break;
			}
			case NSStreamEventEndEncountered:
			{
				
				NSLog(@"input stream: finished downloading");
				
				NSString * str = [[NSString alloc] initWithData:responseData
													   encoding:NSISOLatin1StringEncoding];
				
				NSLog(@"%@", str);
				
			}
				
			default:
				break;
		}
		
	}
}

Thanks for any help in advance

 

[Krevnik]

HTTP is a tricky beast because you aren't guaranteed what the content encoding will be. The headers should be UTF8 safe though.

For something like this, it would actually probably be better to use NSURLConnection instead which will handle the HTTP bits for you and make it easier to build a request and get the response.

 

[turner296]

HTTP is a tricky beast because you aren't guaranteed what the content encoding will be. The headers should be UTF8 safe though.

For something like this, it would actually probably be better to use NSURLConnection instead which will handle the HTTP bits for you and make it easier to build a request and get the response.

Thanks for your reply so soon, I have all the headers as a NSString and have appended the request NSMutableData using the dataUsingEncoding:NSUTF8StringEncoding in the string object. Its really getting me down because why would it work with PERL but not COCOA. I really dont want to use NSURL I need to track the progress of the bits being sent to the stream. Unless that can be done with NSURL framework????

 

[Krevnik]

Thanks for your reply so soon, I have all the headers as a NSString and have appended the request NSMutableData using the dataUsingEncoding:NSUTF8StringEncoding in the string object. Its really getting me down because why would it work with PERL but not COCOA. I really dont want to use NSURL I need to track the progress of the bits being sent to the stream. Unless that can be done with NSURL framework????

Yes, NSURLConnection will provide data from the body as it comes in, and will expose the response headers/status as properties and a dictionary. The thing I see is that you assume the incoming data is ISO Latin. Assuming the strings coming in are the same, any difference would be due to assuming it is latin. Perl by default uses UTF8, I believe.

Using NSURLConnection should make it easier to find out what the content encoding is and use that when you take the body and convert it into a string.

It might also be helpful to see what string you get from perl from the server, and the string your Obj-C code thinks it is getting.

 

[turner296]

Yes, NSURLConnection will provide data from the body as it comes in, and will expose the response headers/status as properties and a dictionary. The thing I see is that you assume the incoming data is ISO Latin. Assuming the strings coming in are the same, any difference would be due to assuming it is latin. Perl by default uses UTF8, I believe.

Using NSURLConnection should make it easier to find out what the content encoding is and use that when you take the body and convert it into a string.

It might also be helpful to see what string you get from perl from the server, and the string your Obj-C code thinks it is getting.

Thanks again for your help but I dont think is to do with the encoding to be honest, when I do the upload in PERL it sends me back the MD5 checksum of the uploaded file, when I do it in COCOA it sends me the NSData containing the servers homepage HTML. Its as if its not sending the data to the right host, or its sent it to the right host but a connection was lost so a redirect from the server has made the inputstream download the homepage. I dont know if you understand that?

if i was going to go with NSURL how would I track the progress of the application SENDING the data, NOT recieveing??

 

[Krevnik]

Thanks again for your help but I dont think is to do with the encoding to be honest, when I do the upload in PERL it sends me back the MD5 checksum of the uploaded file, when I do it in COCOA it sends me the NSData containing the servers homepage HTML. Its as if its not sending the data to the right host, or its sent it to the right host but a connection was lost so a redirect from the server has made the inputstream download the homepage. I dont know if you understand that?

if i was going to go with NSURL how would I track the progress of the application SENDING the data, NOT recieveing??

NSURL is a URL string utility class. NSURLConnection is a class that wraps up URL access & streams into an easy-to-use class. Huge difference.

NSURLConnection uses streams/notifications to send request data and receive response data. That isn't very different.

So now I wonder, what are you sending in each case? You say it is the same thing, but if it was, you should get the same response from the server. So something must be missing in the Obj-C version.

 

[Sayer]

Get HTTPScoop and run the Cocoa app a few times to see what is happening at the HTTP exchange level (actual HTTP connection, response, headers, etc).

It's an invaluable tool when you are doing web-based app development.

The only other alternative to get deep into the networking transactions is to use WireShark on a Windows box to view a 'tcpdump' file from a Mac. WireShark for Mac is a joke, if you can get on a Windows laptop or use BootCamp you can make a tcpdump log file using:

Terminal command:

sudo tcpdump -i en0 -vvv -n -s 0 -w -~/Desktop/DumpFile.dmp

Hit Control-C to stop logging. Move the .dmp file to Windows and open it with WireShark. That will show everything going on down to the TCP networking level.

I had problems talking to a certain DSL modem once using a spec called TR-064, which is just UPnP (SOAP/XML) with HTTP Digest authentication added on. The modem wasn't sending a MIME type header back of 'text-xml'. The UPnP's SOAP library was crapping out without even trying to parse the response as it was not in-spec without the MIME type header.

The modem maker fixed their firmware and sent me a test rev to burn and suddenly it all worked perfectly.

Without seeing the low-level responses I wouldn't have figured out it was such a simple thing that was wrong, and the modem that was at fault not my code. HTTPScoop is your friend.