macOS Sediment? xtra 0x0A after `sed ... < x.bin`

[zeppenwolf]

I couldn't tell from the man page (!) if it was kosher to use 'sed' on a binary file... so I went ahead and tried it, and it works fine.

It works fine as far as substituting old_string for new_string, but the thing that's bugging me is that the new file always has one extra char, 0x0A, tacked onto the end. ( What the heck is 0x0A anyhow, it's not a newline??? )

Is there any way to prevent sed from adding that one char?

 

[subsonix]

I couldn't tell from the man page (!) if it was kosher to use 'sed' on a binary file... so I went ahead and tried it, and it works fine.

It works fine as far as substituting old_string for new_string, but the thing that's bugging me is that the new file always has one extra char, 0x0A, tacked onto the end. ( What the heck is 0x0A anyhow, it's not a newline??? )

Is there any way to prevent sed from adding that one char?

0xA is a newline and sed appends it to the output. Not sure if/how you can prevent that though, you have to look further.

 

[northerngit]

but the thing that's bugging me is that the new file always has one extra char, 0x0A, tacked onto the end. ( What the heck is 0x0A anyhow, it's not a newline??? )

Is there any way to prevent sed from adding that one char?

You could try piping your output to 'tr'. For example,

sed .... | tr -d '\n'

 

[chown33]

Use xxd to convert to hex, then run sed (using hex to designate what to replace), then run that output through xxd to translate it back to binary.
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/xxd.1.html

The -p and -r options will probably be of most interest.


For better or worse, sed is primarily an editor of text, not binary. Thus, its conventions and capabilities are inextricably rooted in text processing. Visit its online man page and see how many times the word newline appears.

 

[zeppenwolf]

Ok, guys, thanks for the help.

One thing I should have mentioned-- it didn't occur to me at the time-- is that old_string is the same length as new_string, (quite deliberately, in fact). So, naturally, the size of the file shouldn't change, which, after all, is how I noticed in the first place that sed was adding one extra char at the end.

So anyway, after thinking forever and googling my buns off, I came up with this:

sed s/MyOldString_${old)/MyOldString_${new}/g <MyBinFile >MyBinFile.tmp
eval $(stat -s MyBinFile)
head -c $st_size MyBinFile.tmp > MyBinFile_${new}

IOW, just get the size of the file, then chop the "new" file to that size...

 

[mrichmon]

A safer (to avoid any ASCII/binary conversion corruption) way to do this binary edit is to use vim in binary mode.

vim -b -c "%s/MyOldString_${old)/MyOldString_${new}/g" -c "w MyBinFile_${new}" -c 'q!' MyBinFile.tmp

The vim command arguments are:

• -b means start vim in binary mode.
• -c "%s/MyOldString_${old)/MyOldString_${new}/g" means after opening the file perform the vim command to search the file for the pattern and replace every found instance of the pattern with the new pattern. This is the same search command that can be typed into vim at the ":" prompt.
• -c "w MyBinFile_${new}" means write the contents of the vim buffer to the file MyBinFile_${new}.
• -c 'q!' means perform the vim command to exit vim without saving the buffer.