macOS cocoa - NSTextView and Terminal Escape Sequences

[DennisBlah]

Hi all,

is there anyone who knows how to parse Terminal escape sequence codes in a string to write it into a textview?
I'm having data that's full of:

[?7l[H[2J[?25l[1;1H[37m[40m  [2;1H  [3;1H  [4;1H  [5;1H  [6;1H  [7;1H  [8;1H  [9;1H  [10;1H  [11;1H  [12;1H  [13;1H  [14;1H  [15;1H  [16;1H  [17;1H  [18;1H  [19;1H  [20;1H  [21;1H  [22;1H  [23;1H

  [?12l[?25h[1;1H

I can't just strip it all, as I asume that [1;1H up to [1000;1H are line feeds...
Please help!

 

[xStep]

Assuming line feeds suggests you don't know your input. Find the documentation and understand it then come back and explain it so others can grasp the source and format. Right now your taking a blind shot that someone will recognize it.

Those lines remind me of VT Terminal control codes. That would likely be non-compatible with UITextView.

 

[DennisBlah]

Assuming line feeds suggests you don't know your input. Find the documentation and understand it then come back and explain it so others can grasp the source and format. Right now your taking a blind shot that someone will recognize it.

Those lines remind me of VT Terminal control codes. That would likely be non-compatible with UITextView.

Hi xStep,

my apologies.
I'm making an SSH shell, as a telnet shell is 1) insecure, and 2) windows shell server is best running on ssh and our cisco equipment is only allowing ssh since last few weeks.

I'm creating an shell in xcode, so I can wrap a gui around it with some usefull functions for user management, server management, cisco management etc.

Currently I'm running the commercial bitvise ssh server on our windows systems.
Now for me this is quite simple to use by terminal, but it's all taking time and building a gui around it would be great.
For our network management I wrote the same kind of applciation.

For some reason bitvise is giving me this output back incl my result, this is a full example of a result of login and dir c:

[?7l[H[2J[?25l[1;1H[37m[40m  [2;1H  [3;1H  [4;1H  [5;1H  [6;1H  [7;1H  [8;1H  [9;1H  [10;1H  [11;1H  [12;1H  [13;1H  [14;1H  [15;1H  [16;1H  [17;1H  [18;1H  [19;1H  [20;1H  [21;1H  [22;1H  [23;1H  [24;1H  [?12l[?25h[1;1HMicrosoft Windows [Version 6.3.9600][2;1H(c) 2013 Microsoft Corporation. All rights reserved.]2;Administrator: Command Prompt[4;1HC:\Users\Administrator.DENNIS>dir c:
dir c:]2;Administrator: Command Prompt - dir c:[6;1H Volume in drive C has no label.[7;1H Volume Serial Number is AA8C-AC49[9;1H Directory of C:\Users\Administrator.DENNIS[11;1H08/12/15 07:50 PM  <DIR>  .[12;1H08/12/15 07:50 PM  <DIR>  ..[13;1H08/22/13 05:39 PM  <DIR>  Desktop[14;1H08/12/15 07:50 PM  <DIR>  Documents[15;1H08/22/13 05:39 PM  <DIR>  Downloads[16;1H08/22/13 05:39 PM  <DIR>  Favorites[17;1H08/22/13 05:39 PM  <DIR>  Links[18;1H08/22/13 05:39 PM  <DIR>  Music[19;1H08/22/13 05:39 PM  <DIR>  Pictures[20;1H08/22/13 05:39 PM  <DIR>  Saved Games[21;1H08/22/13 05:39 PM  <DIR>  Videos[22;1H  0 File(s)  0 bytes[23;1H  11 Dir(s) 56,807,833,600 bytes free]2;Administrator: Command Prompt[24;1H
[24;1H  [24;1HC:\Users\Administrator.DENNIS>

p.s. I already tried a manual parse on [24;1H, but this ends up in a load of line feeds, which are not in my own mac's terminal application.

When I logon with terminal (I have all settings default, so it's white with black text), the window becomes black with white text etc.
No weird codes like above.

So I assumed that this output I get in my application first was ANSI but after some research I found out it is Terminal Escape Sequences. I tried different type of pty pty types: Vanilla, VT100, VT102, VT220, normal Ansi, and Xterm

Xterm gives me the best output, but still with these commands, these are in the data I receive back, and so written to the NSTextView.

Is there any API for this? Or a different approuch? I'd like to support this when it occures.

It should be stuff as text color, background color, text highlight color etc

 

[chown33]

See here:
https://fogbugz.bitvise.com/default.asp?WinSSHD.1.12076.10

Note what they say about how bitvise is being told about terminal emulation. Your SSH client is probably doing something like that, i.e. it's requesting a terminal emulation.

Maybe you can try an SSH server that doesn't try to emulate a terminal.


I also suggest that you carefully read a reference on ANSI terminal escape sequences. You should be able to recognize most or all of them in the output you get from bitvise. Here's a simple reference:
https://en.wikipedia.org/wiki/ANSI_escape_code

You will have to look into the details of the escape sequences thoroughly. That's going to take time.

If there are sequences being emitted by bitvise that aren't ANSI compliant (or are incorrect for the emulated terminal), then please point them out. I didn't notice anything in what you posted that isn't ANSI. Details are important, including the one that each sequence actually starts with an ESC character, which you should at least show as "\033" or something similar. In other words, show every byte in the output you post.


I don't know of any ANSI terminal emulators for NSTextView. That seems like something you could search for.

 

[DennisBlah]

See here:
https://fogbugz.bitvise.com/default.asp?WinSSHD.1.12076.10

Note what they say about how bitvise is being told about terminal emulation. Your SSH client is probably doing something like that, i.e. it's requesting a terminal emulation.

Maybe you can try an SSH server that doesn't try to emulate a terminal.


I also suggest that you carefully read a reference on ANSI terminal escape sequences. You should be able to recognize most or all of them in the output you get from bitvise. Here's a simple reference:
https://en.wikipedia.org/wiki/ANSI_escape_code

You will have to look into the details of the escape sequences thoroughly. That's going to take time.

If there are sequences being emitted by bitvise that aren't ANSI compliant (or are incorrect for the emulated terminal), then please point them out. I didn't notice anything in what you posted that isn't ANSI. Details are important, including the one that each sequence actually starts with an ESC character, which you should at least show as "\033" or something similar. In other words, show every byte in the output you post.


I don't know of any ANSI terminal emulators for NSTextView. That seems like something you could search for.

Hi chown,

My current fix is to turn the default emulation to psuedo only (server-side)

But in order to 'support' servers like this I'd like to strip/parse these codes, as I won't be able to use it in textview anyway.

I was hoping that this already has been done by someone,somewhere but I cant get the right keywords to google it as I dont get my results

I found this
https://github.com/chalk/strip-ansi/blob/master/readme.md

But these codes are not the same as what I'm getting back

 

[chown33]

Hi chown,

My current fix is to turn the default emulation to psuedo only (server-side)

But in order to 'support' servers like this I'd like to strip/parse these codes, as I won't be able to use it in textview anyway.

I was hoping that this already has been done by someone,somewhere but I cant get the right keywords to google it as I dont get my results

From what you've posted so far, it appears that every code starts with ESC-[ and ends with a letter. That seems like a pretty simple parse to implement. See the heading "CSI Codes" in the wikipedia article I previously linked to.

There's a big difference between stripping the codes and actually interpreting them. Stripping is simple: remove everything between the CSI (ESC-[) and the next subsequent letter. Interpreting the parameters and the cmd-codes is quite a bit more difficult, but there are only a small number of cmds in what you posted: 'H', 'J', 'm', 'l', and maybe a few I missed.

I found this
https://github.com/chalk/strip-ansi/blob/master/readme.md

But these codes are not the same as what I'm getting back

How do you know they're not the same as what you're getting back? Be specific: describe exactly what you did to ascertain that they're different.

You haven't yet posted a complete list of every character you're getting back. For example, the posted output doesn't show any ESC characters.

You need to post the complete output, with every character visible, so we can see the exact sequence, including all the control characters. As I said before, details are important. I don't see how anyone can help you solve the problem without knowing the exact details of the character stream.

 

[DennisBlah]

From what you've posted so far, it appears that every code starts with ESC-[ and ends with a letter. That seems like a pretty simple parse to implement. See the heading "CSI Codes" in the wikipedia article I previously linked to.

There's a big difference between stripping the codes and actually interpreting them. Stripping is simple: remove everything between the CSI (ESC-[) and the next subsequent letter. Interpreting the parameters and the cmd-codes is quite a bit more difficult, but there are only a small number of cmds in what you posted: 'H', 'J', 'm', 'l', and maybe a few I missed.


How do you know they're not the same as what you're getting back? Be specific: describe exactly what you did to ascertain that they're different.

You haven't yet posted a complete list of every character you're getting back. For example, the posted output doesn't show any ESC characters.

You need to post the complete output, with every character visible, so we can see the exact sequence, including all the control characters. As I said before, details are important. I don't see how anyone can help you solve the problem without knowing the exact details of the character stream.

I just hooked up the rawData channel delegate from NMSSH and receiving logging this data from the NSData:

RAW DATA: [?7l[H[2J [?25l[1;1H[37m[40m  [2;1H  [3;1H  [4;1H  [5;1H  [6;1H  [7;1H  [8;1H  [9;1H  [10;1H  [11;1H  [12;1H [13;1H  [14;1H  [15;1H  [16;1H  [17;1H  [18;1H  [19;1H  [20;1H  [21;1H  [22;1H  [23;1H  [24;1H  [?12l[?25h[1;1HMicrosoft Windows [Version 6.3.9600][2;1H(c) 2013 Microsoft Corporation. All rights reserved.]2;Administrator: Command Prompt[4;1HC:\Users\Administrator.DENNIS>

It does not matter if I use UTF8 or ASCII encoding... Maybe you have any advice for me?
Again, this seems just to be a nice to have if it works, but then again I would not be able to set text color, text highlight color etc in a NSTextView anyways? I can only the complete NSTextView settings?

 

[chown33]

I just hooked up the rawData channel delegate from NMSSH and receiving logging this data from the NSData:

RAW DATA: [?7l[H[2J [?25l[1;1H[37m[40m  [2;1H  [3;1H  [4;1H  [5;1H  [6;1H  [7;1H  [8;1H  [9;1H  [10;1H  [11;1H  [12;1H [13;1H  [14;1H  [15;1H  [16;1H  [17;1H  [18;1H  [19;1H  [20;1H  [21;1H  [22;1H  [23;1H  [24;1H  [?12l[?25h[1;1HMicrosoft Windows [Version 6.3.9600][2;1H(c) 2013 Microsoft Corporation. All rights reserved.]2;Administrator: Command Prompt[4;1HC:\Users\Administrator.DENNIS>

It does not matter if I use UTF8 or ASCII encoding... Maybe you have any advice for me?
Again, this seems just to be a nice to have if it works, but then again I would not be able to set text color, text highlight color etc in a NSTextView anyways? I can only the complete NSTextView settings?

Does the raw data contain ESC characters or not?

If it does, then why aren't they being displayed? You haven't shown any code, so I can only guess about why. How are you displaying the NSData bytes? Details are important. You haven't really provided any.

If the raw data doesn't contain ESC characters, then why not? Those are crucial parts of an ANSI stream, so where are they? Are they being stripped by "NMSSH"? By something else? Are they appearing correctly in the SSH cleartext but being lost somewhere else?

You will probably need to spend some time looking into details, possibly with a debugger, to figure out where the ESC characters are. If they don't exist at all in the SSH stream (seems unlikely, given that Terminal interprets things, as does Xterm), then find a better SSH server. On the other hand, if the ESC chars are there in the NSData, then you need to work out a way to display them, along with every other non-printing character. When you have it, post that data.