Why my PDA is an iBook, and what the next one will be

[bnewman]

I had a Palm III for a while, but I didn't really use it. Today, I have an iBook, and I use it as a PDA more than I ever did the Palm, because it's a real computer and I don't have to worry about synching it (which never really worked right for me). The thing is, in most situations where I can afford to devote both hands to computing (one for the PDA, one for the stylus), I can afford to sit (or squat) and take out my iBook (I'm typing this on the bus right now). The Palm wasn't more portable in the sense of being more convenient to use, just in the sense of being smaller and lighter, and I have a nice, padded computer backpack and routinely pack my iBook when I'm going anywhere except shopping.

So, what would qualify as more portable than this iBook? A real wearable computer would. I'd like a dataglove (which, now that we have the Wiimote, should be well within reach) and a transparent, binocular heads-up display which places virtual objects at in an arms-length-radius sphere around my head, organizing data using a BumpTop-like spacial persistence model. An Exposé-like show-me-my-desktop gesture moves everything to the periphery so I can see the real world.

The thing is, this doesn't require a new computer — I'm already carrying a computer everywhere that's more than capable of all this. All I need is my bluetooth dataglove and HUD, and an appropriate desktop manager, and I'm good to go. So where are they? The dataglove should be easy after the Wiimote. The HUD will be harder.

Anyway, after we have these the possibilities open up a lot. To the sensor which locates the dataglove and HUD relative to my body, add another that locates them relative to the room. Now I can place objects at fixed locations in space, which would be great for someone like me who likes to pace while developing an idea. I can even save the arrangement of virtual objects and reopen it in another room of the same dimensions. And if any spam gets past my sentient AI secretary, I can crumple it up and throw it into the trash from across the room.

Another killer app would be location-based services — using GPS or, for greater precision, a neighborhood council's localizer grid, I could see Mapquest, Google maps, or Google Earth overlaid on reality. I could click on a restaurant and open a PDF of their menu. This should be enough of a gain for local businesses that it shouldn't be too advertizing-polluted. (I would expect the virtual world to have a bit more advertising density than the real one, but not by that much. This may be hopelessly naive.)

I'd pay $50-$100 for the dataglove and $200-$400 for the HUD, assuming both lived up to my expectations. It should be possible for these technologies to hit those price points within the next 5-10 years... here's hoping.

Comments

[sinsofthedove.livejournal.com]

And if i was reading a PDF of an article that I hated, I could fling it against the wall, just like I do with crap books in real life! Awesome!

[foxfour.livejournal.com]

or through the wall, more likely?

[foxfour.livejournal.com]

here's hoping, indeed.

suggestions for entering text? ideally, it'd be really good handwriting recognition.

[orawnzva.livejournal.com]

I think I'd prefer voice, maybe with a sensor attached to my throat to detect subvocalization so I don't have to speak out loud. A good fully-featured wearable system would include a variety of text input methods, including voice, air-writing, air-typing, and ASL fingerspelling (or even full ASL, if we're getting ambitious).

[foxfour.livejournal.com]

from an NLP point of view, "awww, do i have to?"

seriously, all those would be great. handwriting seems the most easily feasible, though.

(i like the idea of a virtual tablet, on which one can write — air writing to all around you, but to you — a floating scroll! or whatever!

and think about the possibilities with a zeroconf network and sharing objects.)

[foxfour.livejournal.com]

addendum: problem with airwriting and the bumptop-style interface: in both cases, pressure is important. what will the glove measure pressure against? distance from the reference point? it would need to correct for slight movements of a reference point that you carry, and need to allow for wobble in hand-placement against the air. but could be done.

[orawnzva.livejournal.com]

BumpTop just happened to be on my mind, but really an interface involving objects floating in space is going to be very different from one involving objects sliding across the surface.

Anyway, you'd use gestures — make a grasping gesture to pick up an object, open your hand to let it go, etc.

[foxfour.livejournal.com]

that sounds good.

[sildra.livejournal.com]

I know many people who type more quickly than they handwrite, and several others who can't even read their own handwriting. I think I'd prefer air-typing to air-writing. I actually have very legible handwriting, but I write at a pretty average speed, and type almost fast enough to take dictation. Part of the reason I've never been attracted to PDAs in general is I don't like it that you have to handwrite--what's the point of having a computer if you can't type?--and my fingers (way on the small end for an adult) are too big to ten-finger type on those things.

[foxfour.livejournal.com]

points. i've not tried air-typing, but my intuition is that i would have lower accuracy than with real typing or air writing, though.

[orawnzva.livejournal.com]

In the case of air typing or air writing, of course you would have lower accuracy if it were tracking the movement of your fingers relative to a virtual keyboard or page located in space, but it won't be doing that, it'll be tracking the movement of your fingers relative to your wrists. Try making writing motions in the air and see where the information is really being generated.

Of course, only real touch-typists will be able to air type — since there's no virtual keyboard per se, there's no way to hunt and peck. That's why I, who can't type very fast, want voice, possibly augmented with some derivation of Dasher for fast correction/disambiguation.

[tibicina.livejournal.com]

Well, why not have a virtual keyboard? I mean, it's a virtual world of virtual objects - how hard could adding a keyboard be?

[ccommack]

I actually have a mild objection to devoting too much power/hardware to input. Not only do I think it's inelegant, there's a practical limit to how much attention you want people to be able to devote to a computer environment when they're walking around in the world. I think restricting input to a few basic functions (clickwheel on a wiimote) would be fine, and saves the input for keyboard-and-pointer. (Yeah, I'm a CLI partisan. Shoot me.)

[orawnzva.livejournal.com]

I'm a CLI partisan, too, that's why I want voice! You need hands-free text input to be able to take notes in a hands-free situation. I don't want to have to be fumbling for a thing when I'm trying to pay attention to what smeone is saying so I can write it down. Yes, someday someone will get hit by a car because they tried to cross the street while playing solitaire on their HUD. That future person is an idiot. I am not going to do that.

[ccommack]

I'm just unconvinced that voice is a good way to interact with a CLI.

[orawnzva.livejournal.com]

It's not a good way to interact with a current CLI because they aren't designed for it. Current voice-recognition interfaces are either makeshift skins on top of other kinds of interfaces or voicemail mazes where you only have a few choices. A voice interface that is really a voice interface has yet to be developed.

[orawnzva.livejournal.com]

In fact, my GPS-enabled, map-aware computer of the future will beep at me if I try to cross the street while interacting with a graphical virtual object that is not located on the street.

[mneme]

Looks like HUDs are about twice your price point at the moment (assuming you don't insist on head tracking and 3d vision). Other stuff's probably farther, as yet -- though people were doing wearables as early as the mid '90s via gaming datagloves and the extant HUDs.

[mneme]

Actually, the I-Trek (from http://www.vrealities.com/hmd.html) is $200 now, for a 320/240 HUD. But better resolutions are more.

In some ways, most interesting on that site is the Nomad -- 7k, sure, but it's a monocular HUD that projects an 800x600 image ON YOUR RETINA -- having it float above the ambient view without blocking your eyesight! Talk about not interfereing...

That said, it's also monochrome -- which is presumably a function of the useful medium as well as the current tech. One wonders whether this is likely to be worked on long term -- as opposed to full-enclosure video that just digitally merges the delivered image with the ambient one. With fast enough digital processing, there wouldn't be much of a delay for the latter (vrealities does have a few applicatiosns of that; I noticed a nightsight/HUD hybrid), and with a bit of shrinkage, you could manage to inject data into your view without blocking that of the ambient environment at all (shrink it and stick 270 degrees of vision in a box, or put the displayed words on the most visible white surface (actually, you could manage that one with the eye projection tech), or whatever).

[orawnzva.livejournal.com]

Full enclosure video is likely to suffer from lag, eyestrain, be insufficiently wide, and have insufficient dynamic range, long past the point when a transparent HUD becomes feasible.

[orawnzva.livejournal.com]

Yeah, what I want is the next thing after the Nomad — transparent (although you could add a plain black/transparent LCD layer if you wanted to be able to make some virtual objects opaque), wide-angle (none of this 23" monitor nonsense, I want peripheral vision), binocular, and color. Oh, and it shouldn't weigh anything. I guess I might be waiting a while for that to come down to my price point.

[mneme]

heh. That's not so much the next thing as the next thing + N. But that sounds cool, yeah.

[foxfour.livejournal.com]

ah, also:

a design goal of unununium that i'd like to see in this UI: "more interconnection".

(can you tell i really like this idea?)

[orawnzva.livejournal.com]

*looks...*

Fascinating. In Python. I've been thinking more or less that about interconnection for years — I'd particularly like to be able to arrange components into applications that actually do what I want, without having to muck about with the internals of everything. I may join them, if the project is still alive.

[foxfour.livejournal.com]

it seems sadly close to dead. i'd love to help myself, if i had more useful knowledge.

[orawnzva.livejournal.com]

A lot of the features — like interconnection — could be implemented as a highly pluggable application suite, without bothering about the more ambitious OS aspects. I should contact them and see what the state of their code is — Python GUI programming is on my skill-rank wishlist for this semester.

[mneme]

oh, FWIW, on the actual topic --

I have two PDAs, neither of which are a laptop.

I did carry around a 2.5 pound laptop, for years. Still have it; it sits on a shelf and occasionally comes out as a mobile fidget toy or internet platform (depending on how Debian is treating me this year) during gaming sessions. But it doesn't work for me for traveling -- it's too heavy, too bulky, too fragile, has no (2-6 hours) battery life, and actually takes mesurable time to start up or shut down.

By contrast, my Zaurus is quite adequate as a mobile gaming platform (emulators, Nethack, or Angband), programming/calculating platform (perl and gcc), and filk library holder/songwriting platform (good thumb keyboard), whereas my sideckick is excellent as a mobile internet platform, decent enough as a phone, and an amazing communication tool and other (lj, zines, pbem gaming) writing platform. And both stay powered all day, never (hardly ever -- the Z will sometimes need to be plugged in during all night filk sessions during a con, especially if I've been fidgetting on emulators for much of the con) needing to be plugged in.

The sidekick, too, is pretty good for true one-handed operation -- thumb keying with a single thumb is -quite- adequate for quick messages on the go, and two-thumb typing is actually pretty fast (if not as fast as touch).

[orawnzva.livejournal.com]

We should remember to have a fascinating conversation about all this next weekend.

[mneme]

We really should! With visual aids and everything.

[q10.livejournal.com]

two things:
1. you may need a new computer, since a laptop that's constantly being asked to use its hard drive from inside your backpack while you're walking will be at far too high of a risk for catastrophic failure.

2. i've been saying for years that i wanna be a gargoyle, but it's become clear to me that we aren't a big enough demographic yet.

[orawnzva.livejournal.com]

1. three words: "sudden motion sensor"

2. yeah, more's the pity

[q10.livejournal.com]

the sudden motion sensor solution is imperfect, and, more to the point, the version of it that was explained to me works by locking the drive heads which means that hard drive access would be impossible while the sudden motion is in progress, which, if you're wearing the damn thing while you're walking around GPSing, could be prettymuch all the time. there are shock absorption technologies that would address this, but i suspect they aren't implemented at the level you'd want in your current laptop, so at the very least you'd want a line of computers designed with this issue in mind.

[ccommack]

What you'd want to do is run everything you could off an iPod hard drive, while using enough of the tricks the iPod itself uses to minimize hard drive usage. Fortunately, even flash memory is cheap.

[q10.livejournal.com]

oh, there are any number of technologies that can be had pretty cheaply today that can solve this problem. it would not surprise me to learn that somebody has already put all or most of them into a fully functional or nearly fully functional laptop, but, as it is, i'm pretty sure not enough of them are standard, so new computer shopping might still be necessary.

[foxfour.livejournal.com]

not sure that the iPod tricks would work as well for general computing; if i'm not mistaken, a large part of the iPod hard drive's resistance to shock comes from read-ahead and caching in ram. works well for a music player, don't know if it'd help us here. however, flash memory, or something like, might be the ideal route, if cost is not a big issue.

[devinsong.livejournal.com]

Ooh, cool topic. I'm deeply envious, BTW. As a car commuter, I almost never have the opportunity to multi-task while commuting. (Voice data entry would *not* help- I need to focus on *driving*.)

I do have both laptop and PDA. The laptop is more luggable than portable though- a compromise since I'm using it as my primary computer, and the extra wide screen is too damned handy to work with multiple windows.

The PDA is an old one and purely and simply for dumping text into when I *do* happen to have a spare moment (I have a keyboard for this- I don't do anything longer than a few words with a stylus). And occasionally as a calculator, datebook and address book. I'm not at all sure how much I'd use the other functions of a PDA. I suspect not much- if I'm at work or at home, which is most of the time, I have a high-speed connection with a better interface on demand.

I'm fascinated with the HUD idea for applications like the GPS/mapquest interface- it'll be interesting to see if better voice interfaces can get fast and accurate enough to challenge a good touch-typist. I'm constantly reminded of the guy who said (of computers) that the things we thought would be difficult are easy and the things we thought would be easy are hard.

[orawnzva.livejournal.com]

As a car commuter, I almost never have the opportunity to multi-task while commuting. (Voice data entry would *not* help- I need to focus on *driving*.)

What I would want is a voice-command/HUD system (and putting a HUD in a car is a lot easier, because you can project things on the windshield) that is reluctant to let me use anything other than navigational and tactical overlays while the car is moving. By tactical overlays, I mean infrared (for poor visibility conditions), rear and side mirrors, relative speed indications, proximity alerts, etc.