We’re in for some big changes in the area of desktop computing. The desktop personal computer has already disappeared from the homes of many; replaced by a laptop. For the majority, most functions of the laptop can be replaced by a tablet, and it’s easy to use touch based UI. But what about the desktop PC? Is it even needed? Could a touch based UI ever work on a desktop computer?
Who Needs a Desktop Anyway?
Anyone who does real work using a computing device for 8 hours a day. Examples are software development, video editing, graphic design, number crunching in Excel – all are tasks which are made better for using one or more 24″ monitors. Can these users and their applications adapt to a touchscreen world?
The Catch-22 of Touch
There is no doubt that the two major desktop operating systems vendors, Apple and Microsoft, are steering us towards a touch future. Apple are moving OSX into line with the iPad, with the new full screen mode and reverse-scrolling of screens. Microsoft’s Developer Preview of Windows 8 is ‘touch-first’ with it’s Metro-like UI.
These moves don’t mean the transition to touch will be easy. There are significant hurdles to overcome.
The Hardware Isn’t Ready
There are a few touchscreen Windows PCs on the market, but the touch is an afterthought. It is not physically possible to use current touchscreen monitors for any length of time. They stand on the desk in a vertical or near-vertical alignment, at nearly arms length. Using these screens for any length of time will result in increased cases of musculo-skeletal injuries of the upper arm, shoulder and neck.
The Software Isn’t Ready
Applications are written with a mouse in mind. The user interface has changed only slightly since the first windowed operating systems. We see fairly static screens with every conceivable option crammed onto the screen at once. Menus have become ever more conjested, and toolbars of icons have become unusably cluttered. Canonical have seen this problem and ditched the menu from apps in favour of a HUD in Ubuntu 12.04 (YouTube). For a touch device, the icons are just too small and too densely packed. A radical redesign is needed to make complex apps work with a touch UI.
The Users Aren’t Ready
Ditching the mouse will be harder than adopting it (image courtesy http://www.e-tech.ca)
We need to give up our beloved mouse. But this move away from the desktop rodent is going to be a bigger shift than the move from text-only PCs to windowed operating systems in the late 80′s. I can still remember having conversations with a die hard SuperCalc user who was convinced that colour screens, the mouse and a windowed environment brought no benefits, and that they wouldn’t catch on. Now he happily uses colour to highlight specific data in Excel, and opens windows side by side, and drag and drops with the mouse. Similarly, there are many users now, who believe the mouse is here to stay.
So, for desktop touch to catch on, the hardware and the software need to be ready, but manufacturers are not keen to invest in the area until desktop touch catches on. But users will never adopt touch without much better designed hardware and software. Catch-22.
As a result we’ll likely see sluggish adoption of Windows 8, once users realise it is the worst of both worlds. Just like Windows 1.0 and 2.0 faced resistance during the text-only years, it will not face mass adoption.
Putting the Desktop back into Desktop Computers
vintage drafting table
It doesn’t take a lot of imagination to picture a 30″ x 40″ multi-touch screen in place of the top surface of a drafting table. The key feature of these tables is their adjust-ability. They can adjust for tilt. They adjust for height, often with a big enough range to allow for working sitting or standing. This is a screen you can rest your forearms on when required (without the touch screen going crazy).
Optimus Popularis Keyboard
For plain text entry, nothing as yet beats a real keyboard – so I envisage an ergonomic keyboard of around 70 keys, built into the lower edge. The keyboard would not tilt with the rest of the surface. The less used options of a traditional keyboard will be put on the touch screen. Ideally the keycaps of the keyboard would each be a mini LCD or eInk display like the optimus popularis, so they can adapt to the application’s needs. Despite the naturalness of touch, keyboard shortcuts will feature heavily in the OS and applications. Keyboard shortcuts are already making a comeback on the web, and will reduce the amount of arm waving for many tasks.
Such an integrated desktop solution will likely need to come with a chair designed for this new way of working. We’ll be much more mobile at our ‘desks’. Gone will be the micro-movements of the mouse and it’s single slow pointer. When we can use both hands, we’ll need to be able to comfortably lean forwards in our seats to reach the top of the screen, or lean back and use the keyboard. Standing up may become a normal way of working. Drafting chairs are often no more than bar stools for perching rather than the huge padded ‘executive’ style chair we see today. This extra movement could be a good thing for bodies too.
The Painful Transition
Until we all have one these truly ‘desktop’ computers, there will be pain.
There will be physical pain we try using a classic touchscreen monitor instead of the mouse.
There will be mental pain flitting back and forth between designed-for-touch apps and designed-for-mouse programs on the same computer. Just try the Windows 8 preview if you need convincing of that.
There will be pain for software developers as they realise they have to do real usability testing as part of their design. Lumping every conceivable program option into a menu and onto a toolbar is not design. They will need to design fluid, context aware user interfaces that constantly adapts to what the user is doing. It’s been done on various iPad apps. The challenge is the big programs such as Photoshop, Visual Studio, or Final Cut Pro. It’ll need a lot of thought.
There will also be pain for hardware manufacturers who lose sales to tablets, and are too fearful to innovate in the design department.
Apple to the Rescue
I believe the only company with the guts to pull of this kind of computing revolution is Apple. A future iMac could be more iDesk. A large piece of furniture with the styling of an object d’art and a heart of silicon. The whole experience will have to be carefully designed – exactly what Apple is so good at.
As long as they haven’t tried to patent the design of a drafting table (which wouldn’t surprise me), others will follow – cautiously at first, and then we’ll see generic ‘desks’ able to run Windows or Linux. Different shapes and sizes will appear. Geeky ones with hidden LED arrays, flat pack IKEA inspired styles, retro and vintage designs. These will need to be upgradable – and we may unfortunately see a return to proprietary upgrades to keep revenue flowing.
It will be interesting to see how this pans out. But I suspect it will be a longer time frame than the change from text-based operating systems to windowed.
The PC may disappear from a great many homes, but in our places of work, including our home offices, the next few years are going to see a revolution in the design of the desktop PC – and it’s long overdue.
A couple of years ago I bought a PICKit2 for the princely sum of £5 including postage. It was a special offer from Microchip that I couldn’t refuse. I’d tinkered with the Atmel AVR chip in the past, but liked the on-board USB features of the Microchip PIC18F range. I knew it would be useful for some custom HID devices for Flight Simulator one day.
Anyway, I’ve got a simple alarm clock idea which wakes you up by gradually increasing the brightness of an LED strip light behind the headboard of the bed. As a minimum, it needs a simple microcontroller, a real time clock (RTC) of some sort, one or more power transistors for the LEDs, and an LCD screen and buttons for the UI. If I wanted to get this thing done quickly, I’d probably go for an Arduino solution – there are less hardware options and more really good demos and tutorials. However, I already have this PICKit2, and learning the raw nuts and bolts of the PIC range will allow me to avoid the cost of a £20 Arduino in every project.
The Hardware
- PICKit2 programmer, which I believe has ICD capabiities
- 44-pin demo board. This has a PIC16F887 onboard, some LEDs and a potentiometer
The Software
Installed in this order
- Java SE Runtime version 6 (32 bit version, despite being on 64bit OS) (version 7 doesn’t seem to work with MPLAB X)
- MPLAB X from Microchip – don’t download the compilers from this page, they are never the most up to date
- HI-TEC C compiler for PIC10,12,16 (version 9.83 as of Feb 2012, signin required)
- HI-TEC C compiler for PIC18 (version 9.80 as of Feb 2012, signin required)
- Microchip PICKit2 programmer (latest is v2.61 in Feb 2012)
First Test
Although, I want to be writing my own code in C, to test the environment is working, I will first send a pre-compiled HEX file to the demo board, and test.
- Hit the “Auto Import Hex + Write Device” button
- Choose a demo file from C:\Program Files (x86)\Microchip\PICkit 2 v2\DBE Demo\16F887Demo.HEX
- The programmer should erase and then program the chip, but you won’t see anything happen
- You must manually supply power to the board using the ‘VDD PICkit2′ ‘On’ checkbox
I must have spent 2 hours programming and reprogramming the device wondering why the demo wasn’t running. Eventually I tried the ‘VDD PICKit2′ ‘On’ option, hoping it wouldn’t fry my board, and it worked instantly. I’ll put it down to bad UI design of the PICkit2 Programmer – it’s clearly designed with Microchip engineers in mind, and not Joe Public. The help file, whilst technically accurate, needs to spell it out clearly: “Once you have programmed your demo board, you need to supply it with power. The PiICkit2 can do this for you, but you need to tell it using this option [big arrow pointing to diagram]“.
Other pre-compiled demos can be found on the PICkit2 CD-ROM, in the 44Pin Demo Board folder.
The second test (in the next post) is to compile a sample ASM in MPLAB X, upload to the demo board, and run it.
So… I tried what feels like a hack to get my MP4 video to open in VirtualDub.
Two problems now present themselves
- Although I can see the first frame, and can skip to any frame in the file, I can’t play it. This in itself shouldn’t stop me, but I find it so irritating, that I can’t bring myself to use this AviSynth + VirtualDub method
- I can save the audio track from VirtualDub, but it is unusable. When I open it in Audacity, it plays at something like 10x normal speed. I haven’t got to the bottom of what is wrong there
So, I am reluctantly going to re-encode clips I want to use. This is a problem because:
- It takes time. Encoding is several times slower than realtime on my aging Athlon XP processor, and worse on my Intel Core 2 Duo
- There is an inevitbale loss in quality – though whether I can see this on video that has effectively been taken on a pinhole camera, remains to be seen
Currently I’m converting using the SourceForge hosted project: MP4Cam2AVI Easy Converter - which I suspect was born from someone else’s frustrations at trying to accomplish the same thing.
I will re-encode using “XVid MPEG-4 TV HD 16:9″, and PCM (uncompressed audio)
I’ll report on results.
I went snowboarding last week. I took along a £99 Samsung WM200 pocket camcorder. I didn’t take many clips, and they weren’t that good, but I knew they could be improved by shortening and splicing a couple together to form a short sequence. I’ve use Premiere Pro a little in the past, and although it did everything I wanted it to, nothing was ever intuitive (like most Adobe products). So I decided to suppress some of my creative urges and use the simplest app I could find, Windows Live Movie maker.
For the quick edit I had in mind, opened the Samsun W200 MP4 files, and worked well. But there’s always just one more tweak! This time it was the sound. The built in mic picked up a lot of background music outside the mountain restaurant where I’d interviewed one of my ski buddies. All I needed to do was put the sound through an equaliser to remove the bass, and boost the gain. Unfortunately, WLMM doesn’t have such an option. I’d need to extract the audio, tinker with it in a free audio editor, then re-add to my video project, hoping the lip-sync was still ok. The problem is that none of the many utilities I tried would open the MP4 file. Odd, since it played fine in Media Player, Zune, and Movie Maker. The application I really wanted to open it up in was VirtualDub. I don’t even know for sure that I can extract the audio using this awesome utility, but I’d be very surprised if it couldn’t be done somehow.
So the next part of my mission came to be opening the MP4 file in VirtualDub.
After installing lots of apps and codecs, and failing to make progress, I stumbles across this YouTube tutorial youtube.com , and followed the steps. For the sake of my own future sanity, I’ve noted them here.
- Start fresh – uninstall as many codec and video editing apps you can find
- ffdshow
- virtualDub
- Quicktime
- Codec packs
- Install a fresh new copy of VirtualDub
- Install the K-Lite Codec pack, both 32bit and 64bit versions (I have Win7 64bit)
- Leave the configuration until later
- Install updated ffdshow MPEG-4 codec, both 32bit and 64bit version
- Configure ffdshow (make sure H. 264/AVC is assigned a decoder)
- Install AviSynth (this is the important bit)
- For each video file you need to open in VirtualDub, create a .AVS script file
- In it, simply add the following line:
- DirectShowSource(“fullPathAndFilename.mp4″)
At first I thought that creating this script file every clip was a bind. Then I glanced over the documentation avisynth.org
You can script your video editing! As a programmer this is awesome. I could source control my edits. If I decide to swap editing software, I don’t have to start everything again, as my filter settings, in and out points, andf likely a ton of other stuff is safe in my text file.
I’ve yet to explore this properly – this is one evening’s exploration, so there should be more to come.
I’ve been attempting to get some WebSockets code to work using SuperWebSockets and a simple html + javascript page in Chrome. My success has been limited to say the least. To add to the frustration, there is no easy way to examine the TCP data of the WebSocket stream. I use Fiddler extensively when developing web apps, but it is no use for WebSockets – at least not at the moment. Although some support is present – from what I can tell, you can only see the headers, not the stream.
It doesn’t take much googling of this problem before you are told to go and use Wireshark. Although I’ve come across this network packet analyser before, I’d always quickly given up on it. This is not a reflection of the software, but of my impatience in getting to know enough of it to be usable. For WebSockets work, it seems I’m going to have to bite the bullet.
First Problem: Localhost doesn’t have a network interface
Why is this a problem? Because Wireshark needs a network interface to capture packets from. This interface is provided by your network card. Unix systems provide a ‘fake’ loopback network adapter for localhost. Windows doesn’t.
The solution is to add an entry to Windows’ network routing table, so that traffic sent to your local ip address is routed to your network gateway, which will send it straight back to your ip address. Since the traffic now goes through your network adapter, Wireshark should be able to see it.
Using Netcat to test the simplest case possible
To test this out, I will use netcat for windows . This utility allows us to set up a server which will listen on a port and display any text it receives in the console. The same utility also allows us to send traffic to this same port.
The easiest way to do this is to open two elevated command prompts.
In Prompt 1 – set up the server
nc -L -p 8111
-L mean listen, -p is port number
In prompt 2 – Create a Route
route add<your ip> <your gateway ip>
e.g.
route add 192.168.0.16 192.168.0.1
Open Wireshark, add a filter
tcp.port == 8111
In prompt 2 – Connect to the server
nc <your ip> 8111
type some stuff
You should see your text input appear in prompt 1, and see those packets captured in Wireshark. There you will see 3 entries for each request. The first is the original TCP request FROM your ip TO your ip, the second is an ICMP request for a redirect (the routing table at work). The third is the new TCP request FROM your gateway ip TO your ip.
Note that the route will not persist reboots without adding the -p option
So, that proves the network route works.
Back to WebSockets
Problem is, it doesn’t work with websockets.
I set up SuperWebSocketsServerWeb, the sample web app from the SuperWebSockets project. I used iis to host it.
- With the loopback route deleted…
- browsing to http://<local ip>/SuperWebSocketWeb … the web site doesn’t work
- browsing to http://localhost/SuperWebSocketWeb … the web site works, the connection opens and I can use the chat demo – though nothing is captured in Wireshark
- With the route added…
- browsing to http://<local ip>/SuperWebSocketWeb … the web site works, the connection opens, a few packets are captured in Wireshark, then the connection closes a few seconds later.
