Give me Multi-Touch or Give me Death (or really bad carpal tunnel)!

[lookoverthere]

Hello Everyone,

Like many people, I am waiting for Apple to create a fully functional Multi-Touch computer. Over the past year I’ve been just a little obsessed with the topic. It Started when I was sitting in my BioChem class and realized how the current keyboard and mouse interface limited my ability to effectively use my MacBook Pro in class. After a month or so of dreaming about how a Multi-Touch computer would work I realized how complex it would be. So, over the past year I developed a concept for a workable interface and made a few videos and concept art to show off parts of the concept. Please watch the vids (7 in all) and take a look and the pics (around 25 of them) then let me know what you think.

What do you want in a Multi-Touch Computer? Do you even think Multi-Touch Computers would be worth it?

http://macslate.blogspot.com/
http://www.youtube.com/thenewtouch
http://www.flickr.com/photos/thenewtouch/


Thanks

 

[Peace]

You do have too much time on your hands but I must say. Excellent job!!

 

[Marble]

I can tell you put a lot of work into this. Nice.

 

[ravenvii]

Wow, that does look cool.

Apple should definitely take a look at this. Or hell, hire you

 

[liptonlover]

I'm only partly into the third video so far, but I have one question.
One of your goals is to make the most of the screen real estate available, right? So why move the application menus, usually using the otherwise blank space in the menu bar, to the top right corner of the application window? Not only is that a way to waste space, it also requires one more tap to get to a given item because you have to tap just to open the list of menus.

 

[modular]

watched a few videos... really nice work there

 

[twoodcc]

i watched the videos and looked at the pics. great job! i really hope apple takes a look at your work.

 

[lookoverthere]

The Menu Bar/ Command Bar

Hey everyone,

Thanks for the comments so far. greatly appreciate them.

I'm only partly into the third video so far, but I have one question.
One of your goals is to make the most of the screen real estate available, right? So why move the application menus, usually using the otherwise blank space in the menu bar, to the top right corner of the application window? Not only is that a way to waste space, it also requires one more tap to get to a given item because you have to tap just to open the list of menus.

Unlike a keyboard and mouse, which is limited to one window at a time, a Multi-Touch computer would be capable of allowing a person (or multiple persons) to controls several windows simultaneously. At this point the concept of the "On Top" or Active Application disappears. Who would get the menu bar when there are two different windows from two different apps being used? I moved the Menus to the window so that every window is its own self sufficient island. The Menu Bar at the top of the screen is now the Command Bar. The Command Bar simply controls Slate and everything on the screen. The Command Bar adapts to the four user environments: 1) The Desktop. 2) Full Screen Apps (where the menu does return to the top because that window is now the focus). 3) Split Screen Mode (which is really just a way to tile selected windows on the screen). 4) Tabletops (Tabletops are really cool. They are an adaptive environment meant for viewing various files and/or allowing for easy Multi-User access on very large displays. Imagine iPhoto using this environment for a Light Table). So with all the different ways you can interact with and view your apps and files it made sense to me to limit the Command Bar to controlling the environment using the screen instead of the individual apps.

Thanks for you question liptonlover, keep them coming.

 

[twoodcc]

wow, you have really thought this out. i would buy one!

 

[liptonlover]

Another question; what happens when two people are using different applications on the same device/screen? First, how does the system decide who gets more juice if it's running to the limits, second how does the system do the whole highlighting the active window? Which one covers the other when they overlap?

 

[iMacmatician]

The best quote on your flickr page is:

Several friends like to repeatedly remind me of the big names already working in Multi-Touch (Perceptive Pixel, Microsoft with their Surface and Windows 7, Apple, etc.). They say, “why spend all this time working on this stuff when the professionals are working on it? Besides, you don’t even have degree in Computer Science.” It’s because I don’t see anything like my ideas out there yet. And, to my advantage, without a degree in Computer Science I don’t know nor am told how computers are “suppose to be.”

Your videos look very good, some nice concepts there. I have also been thinking about something using multi-touch, but it's not related to your concept.

 

[twoodcc]

i am just curious, but what did you use to make the videos?

 

[lookoverthere]

Apps I used...

i am just curious, but what did you use to make the videos?

I primarily used Blender 3D (free app) to build the 3D models and create the animation, and Inkscape (another free app) to create the graphics. I also used iMovie, Garageband, and Pixelmator.

I really love Blender. http://www.blender.org/ They have a great forum for learning how to use it http://blenderartists.org/forum/ (just be careful on the forums, they really really hate it when people don't search for topics before posting).

thanks

 

[lookoverthere]

Multi-User in some detail

Another question; what happens when two people are using different applications on the same device/screen? First, how does the system decide who gets more juice if it's running to the limits, second how does the system do the whole highlighting the active window? Which one covers the other when they overlap?

There would be two systems in place to handle multi-user work. The first one is called Dynamic Space and the second is Dynamic Area.

Dynamic Space is simple: on a large screen a clear Space (with borders and boundaries) is defined for each user (on a smaller screen the Split Screen environment discussed earlier can be used for this). The users are limited to their respective space. This is the least used system.

With Dynamic Area the OS takes it upon itself to see clustered areas of activity that relate to different users and then define an invisible area around that activity to prevent gesture altering commands from effecting other users outside that area. Remember: I call it "Dynamic" because it adjust to the activity, content, and timing.

For example, let's say you have two photographers using a large light table app. People naturally like to space themselves from others while working, so photographer A and photographer B are keeping to their own sides of the display with their respective pictures. However, photographer A decides to activate a selection command that would cause a 1 finger tap and drag gesture to become a selection tool. With Dynamic Area the OS recognizes that there are two users on the same display and recognizes the areas they are working in. The OS then limits the selection command so that it only effects gestures taking place within photographer A's area.

Because the OS is actively involved in recognizing multiple users it would also be capable of identifying the first, second, third etc. users to the display. So, the first user would be awarded more system resources if the need should arise.

"Omni-Active." That's how the OS looks at all windows. They are all capable of receiving input as long as they are literally visible on the screen. This is because all input to a window has to happen "on" the window. This is another reason why the Menus were moved to the windows themselves. Every window has to be self sufficient. The only commands that you see outside the window would either apply to all windows on the screen or to all windows from one app on the screen.

Now, to the overlap issue. Obviously, Multi-User use works best on larger displays (24in+) so overlap doesn't become an issue. However, Dynamic Space and Area help out here too. The first user is the top user, and any windows associated to that user are all on top. Windows from other users would be overlapped by the first user.

For an individual user a window typically jumps to the front when a gesture is applied to it (unless you are in the middle of applying a gesture to an overlapping window).

Thats the basics. There are a few scenarios where there would need to be tweaks made in the systems, but overall that's it.

Lovin' the questions! Thanks

 

[seven7seven]

oh my god

 

[twoodcc]

I primarily used Blender 3D (free app) to build the 3D models and create the animation, and Inkscape (another free app) to create the graphics. I also used iMovie, Garageband, and Pixelmator.

I really love Blender. http://www.blender.org/ They have a great forum for learning how to use it http://blenderartists.org/forum/ (just be careful on the forums, they really really hate it when people don't search for topics before posting).

thanks

thanks for sharing! i'll have a look at that stuff when i get a chance!

 

[lookoverthere]

???????

oh my god

Is that a positive response or negative?

 

[twoodcc]

Is that a positive response or negative?

i hope positive

 

[TuffLuffJimmy]

I do enjoy the videos, but I think you need to go outside, OP.

 

[lookoverthere]

Outside? What's that?

I do enjoy the videos, but I think you need to go outside, OP.

Well, because I'm in grad school I have to spend several hours in the library. So, I have a lot of time during study breaks to work on this idea. I think I remember what the outside looks like... It's that stuff I drive through from home to school, right?

 

[lookoverthere]

Other Multi-Touch Possibilities

As I have already said above, I believe that Multi-Touch can be used as the main interface system for post-pc computers (Mac replacements basically). But there are other possibilities to mention:

There are several people who don't believe that Multi-Touch would be a good alternative for controlling a Mac-like device, but they would love to see Multi-Touch applied to a larger iPod Touch like device (Newton 2.0).

Still others believe that Multi-Touch is really only good for very very large displays (6ft +) with specialized software only usable by graphic artists, engineers, and storyboard production.

I personally believe that all three applications are possible and each would have a market. I think the post-pc era will bring several devices for our use. Ones like our iPhones and iPod Touches, ones like Macs, and others that have no comparable category as of yet.

Any thought?

 

[liptonlover]

I think future computers should support every modern form of input, from physical mouses and keyboard, to touch/multi-touch, and everything else. Multi-touch is fun, it looks slick and futuristic, but it's not always the best choice. I'm not going to play my favorite desktop fps using just multi-touch. Or any. That works on a smaller device that's completely in your hand, but not for something in front of you/on your lap.

 

[lookoverthere]

Multi-Touch Games

I think future computers should support every modern form of input, from physical mouses and keyboard, to touch/multi-touch, and everything else. Multi-touch is fun, it looks slick and futuristic, but it's not always the best choice...

I would agree with you on this point, to a degree. There are some things where a mouse would work better. But, the place of the mouse would change on a Multi-Touch computer. The mouse would not be available system wide, but would be contained within the apps that needed it (which I believe would be few).

The keyboard to a multi-touch computer would be like a Wacom Tablet to modern computers. You would not see them often, and usually the ones who have them are on a pro or semi-pro level (like writers).

...I'm not going to play my favorite desktop fps using just multi-touch. Or any. That works on a smaller device that's completely in your hand, but not for something in front of you/on your lap.

First Person Shooters are a tricky thing for Multi-Touch. I worked out a basic set of controls for RPGs like WoW, but FPSs are very different.

I think I'm getting there though. I'm currently working on a Multi-Touch Keynote video, so after that I'll try to show some of my ideas for Multi-Touch RPGs and FPSs.

If done right I could see FPSs working very well on 13in+ size displays. Not just on small handheld devices. I have found that the iPod Touch FPSs aren't as fun because the screen is too small. 13in+ displays would provide more room for your fingers to work.


Thanks again!

 

[windywoo]

Very nice work, but why are you posting it here? Apple's spies are everywhere! They will either steal your ideas or sue you for stealing theirs.

On a more serious note, it looked like the viewing screen shrank when the keyboard was being used. I know that could be a problem for some people who like a nice overview of their work.

Overall I thought it looked really nice, tablet PCs have never really been very successful, but perhaps multitouch would be the answer. I'm sure you have seen the posts here about a rumoured Apple tablet as an answer to the netbook phenomenon?

 

[lookoverthere]

Adaptability...

Very nice work, but why are you posting it here? Apple's spies are everywhere! They will either steal your ideas or sue you for stealing theirs.

No, Apple's not Microsoft. I'm sure they'll contact me if my idea is actually good. Apple has too much culture to its products and philosophy to steal an idea like this.

Of course I would love it if Apple hired me to work on this idea with them. But, I don't think that will happen. Maybe I should email Steve Jobs (>>> Apple's in house team of assassins (the iNinjas) jump in their invisible jets (iJets) to come and take me down at the mere thought of a mortal emailing Steve Jobs)

On a more serious note, it looked like the viewing screen shrank when the keyboard was being used. I know that could be a problem for some people who like a nice overview of their work.

You are right. The Full Screen keyboard is more meant for people who need to focus on what they're typing and ignore everything else. The in-window keyboard is good for light typing when multi-tasking.

One of my future videos will focus on multi-display use. Imagine being able to setup multiple touch displays all around you. Each display having its own desktop, but all being connected to your account. If you needed to use the Full Screen keyboard for typing just send that document over to another display. That way all your other windows are still in view on the original display.

Overall I thought it looked really nice, tablet PCs have never really been very successful, but perhaps multitouch would be the answer. I'm sure you have seen the posts here about a rumoured Apple tablet as an answer to the netbook phenomenon?

PC Tablets failed because they took the current Keyboard-mouse centric OS and added a form of touch on top. And it wasn't even a good form of touch. It was single-point stylus based touch. That's why they failed. That's the same reason why Windows 7 touch will fail. Yes, it's multi-touch, but it's on top of that keyboard-mouse OS again.

My concept isn't taking the Mac OS as it is and cramming Multi-Touch in. I'm saying we should completely abandon the Mac OS (all but the most basic components of it), and make a Multi-Touch centric one.

I believe the rumored iTablet is more of an iPod Touch-level device than my proposed Mac-level device. I think it will still be very cool.

Thanks for your encouraging words and thoughts!