Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

SenorClippy

macrumors newbie
Original poster
Jan 21, 2022
3
4
Portland OR
I recently published a blog post for my work on how to remotely control and mirror an iOS device from your browser using a Raspberry Pi and two publicly available github libraries. An Apple developer account and Xcode are NOT needed.https://plusqa.com/2022/01/18/control-your-ios-device-over-the-internet/

It turned out to not be that complicated.The first library, ios-screen-mirror, is a Go program that mirrors a usb-connected iOS device via TCP.The second library, keyboard_mouse_emulate_on_raspberry, is a Python program that emulates a Bluetooth mouse and keyboard on a Raspberry Pi, which is then connected to the iOS device via Bluetooth.Then it was just a matter of sending the browser mouse and keyboard events to the Raspberry Pi (I used Flask and Socket.io for that)

https://senorclippy.tumblr.com%2Fpost%2F674041796561633280
Let me know what y'all think!
 

SenorClippy

macrumors newbie
Original poster
Jan 21, 2022
3
4
Portland OR
This is interesting. But what's with the timer?
It's specific to my use case. The company I work for, Plus QA, primarily does manual testing of apps and websites on real devices. The motivation for me setting this up was to allow testers to test bug regressions on certain edge case devices without physically having the devices. We also didn't want people hogging the remote devices longer than needed or forgetting to end their session so we added a one hour timer that automatically kicks them off lol.
 
  • Wow
Reactions: duanepatrick

prime17569

macrumors regular
May 26, 2021
207
524
This is cool! It reminds me of appetize.io, which I believe uses a similar method to what you're doing, but with iOS simulators as opposed to real devices.

How did you get iOS to work with mouse and keyboard events? I thought only iPadOS supported keyboards and mice.

Also, if you hold down the command key in an app, does a menu bar pop up as it does on iPadOS?
 

SenorClippy

macrumors newbie
Original poster
Jan 21, 2022
3
4
Portland OR
This is cool! It reminds me of appetize.io, which I believe uses a similar method to what you're doing, but with iOS simulators as opposed to real devices.

How did you get iOS to work with mouse and keyboard events? I thought only iPadOS supported keyboards and mice.

Also, if you hold down the command key in an app, does a menu bar pop up as it does on iPadOS?
Thanks!

The Raspberry Pi is emulating a bluetooth mouse and keyboard and is connected to the iphone as such. The keyboard and mouse events from my browser are being sent to the Raspberry Pi where they are converted to hex commands that the emulator uses as traditional mouse and keyboard commands. So to answer your other question, the command key would do whatever a normal keyboard would thats connected to an ipad.
 

prime17569

macrumors regular
May 26, 2021
207
524
Thanks!

The Raspberry Pi is emulating a bluetooth mouse and keyboard and is connected to the iphone as such. The keyboard and mouse events from my browser are being sent to the Raspberry Pi where they are converted to hex commands that the emulator uses as traditional mouse and keyboard commands. So to answer your other question, the command key would do whatever a normal keyboard would thats connected to an ipad.
Thanks for the detailed explanation!

Do iPad keyboard shortcuts work on iPhone (e.g. command-space on the home screen brings up Spotlight)?

For the command key question, I was referring to how a menu bar pops up on iPadOS 15 or newer when you hold down the command key in an app that supports keyboard shortcuts, and asking if this behavior also occurs on iPhone with a hardware keyboard connected.
 

Jakexb

macrumors 6502a
Mar 18, 2014
798
1,106
This is amazing, thank you I've been looking for a way to remotely help my mom sort out issues with her iPhone. I'm definitely going to give this a try. Nothing else has been even close to solving my problem.
 

cslewis2018

macrumors newbie
Jan 4, 2018
2
0
Can this technique work, but with the Raspberry Pi emulating a physical keyboard / mouse via it's USB OTG port?
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.