11/3 Update – ClayAR v0.006SE with (some) icon’s NOW available for download

N.B. Old users, the mode icons show on the current mode (not the other way around). Save is only enabled when you authorize your device (for the folks who don’t want me tracking their devices.)

So, this old nearly-abandoned project is brought back to life(-ish) again thanks to HTC! It’s been really exciting to see how precision “touch” mesh manipulation is possible through a smaller touch point from a stylus!

Note: this is a prerelease available for temp download. Working on the final version’s icons and screen size considerations.

Beyond sculpting… this version includes mesh painting! And fun global operations.

• Edit Mode – pinch in/out to change brush size
  • Paint mode options let you choose colors to paint!
  • Sculpt mode options (coming soon) show different ways to sculpt the clay!
  • Toggle between these modes…
• View Mode – pinch in/out to zoom the mesh
  Global Manipulations – Select an option, then use the slider to select strength of manipulation
  1. Stretch in X Y Z directions
  2. Quadrify
  3. Twist

New way to rotate the clay: two-finger twist, with thumb stationary, and index finger rotating.

Note – the ICONS are UGLY (for now)

With HTC Pen Support (a defunc menu shows up when you press the bottom button of the HTC Scribe Pen)!

I presented ClayAR as having both a software and hardware component, with the software component inspired by the anthrohistorical relation of humans with clay as a creation motif. The crux of the software matter is having to “condense” a relatively complex array of tools for 3d mesh manipulation for usage on a mobile device – yet, without making the version on mobile too difficult to use, or too weak in capabilities. As set by manufacturers’ precedents, a mobile device is essentially a different beast than a desktop computer – as a baseline standard, it has a touchscreen interface, and an accelerometer and gyroscope and camera built directly into its form factor. ClayAR uses the accelerometer and augmented reality as two different ways in which to intuitively control the object’s physical rotation and/or camera zoom/pan/rotate location.

As might be expected for a hobby pet project, the hardware component is still on the drawing board, but…

Here are my slides… an earlier version is available for download to run on your Android (2.2+) or iOS (iPhone, iPad) device:

The complete picture goes full circle, empowering the mobile maker with a device that can 3d holographically project the mesh – or fab it, on the spot…

The ClayAR phone has a 3d holographic projector and a detachable base charger that lets the phone fab itself

Back to ClayAR… One interesting idea came from Jeff Weitzel, who suggested using more than two fingers for other motifs… On the iPad, 4 fingers let you swipe around in multitask, and three finger’s a crunch. For non-AR view, though, three finger toggle could be useful for switching to camera mode (i.e., pinch in/out to zoom cam, and single-touch to dolly the cam). This is to distinguish from physical manipulations to the mesh in multimode.

• Three fingers single touch simultaneously toggles on camera mode
• Three finger swipe left could be undo, Three fingers swipe right could be redo

Some interesting ideas from the meetup for ClayAR -
Haptics feedback so you can “feel” the clay modeling (looking forward to trying out the Immersion Motiv SDK for this).
Showcase app for a highly precise 3d camera
Angel funding for such an app? (yeah right.)

Thinking of releasing it as ClayAR Lite Preview. Two things that are relatively straight-forward to implement to turn this into a free app with in-app payments:

• There will be a tutorial mode that’s kind of like GuitarHero for sculpting… Learn to draw (sculpt) “step by step” by first pressing here, and then here, and then here. Users might be able to submit their own “tutorial mode recipes” to share how to create certain sculpts.
• in-app “marketplace” where people can exchange their creation.
• thinking about charging per .obj download uuid save on the cloud…

(Numbering might actually be v0.003)

The big caveat so far is that I still haven’t figured out what’s making these ARM processors die whenever I try to apply localscale operations dynamically per frame. It works quickly on non-ARM-based lowend devices like my first gen netbook’s (single core Intel Atom oldstuffs).

There was a good mix from different sectors of the games industry — from recruiters to in-game monetization people to graphics designers and even Unity developers(!). The first two might be interesting for ClayAR as a service. It’s fairly simple runofthemill backend work to setup a microeconomy where people can trade and sell their clayar sculpts, for others to expand upon, or even take into their existing pipelines (or print). Who knows, one day, recruiters might recruit people to sculpt clay. o.O

Graphics designers/developer provided some feedback def. to be incorporated in upcoming builds:

Known Bugs: Gotta fix the localScale bug in View mode that froze (perceptively!) the zoom out/in. AR Cam/toggle second toggle (from default gray cam to AR) crashes on Snapdragon devices.

UI finetuning for iPad and tablets and other interfaces – make sure sliders are more easily accessible. Especially accelerometer strength slider.

Sensor Extremes: adjustable and tunable per taste

Materials and Shaders – people seemed to like the toggle, but on the iOS devices, difference in GPU lighting made the nice-looking shiny shader on the Android look with alienscopic lights (think SL with facelight 5000). Lighting control and Materials lab to be added in v003 or v004 (actually rather easy, just need to figure out how to make such an interface intuitive, to keep ClayAR Lite accessible to casual users).

—

March 17: At the ERT Dinner, Max Skibinsky and others suggested expanding the ClayAR marketplace so that people can take what they create and apply simple game “behaviors” to it, such as the ability to AR kick a weird ball they sculpted. Gamification to creation with a tutorial mode that teaches you where to press where to lathe a simple clay cup (similar to step-by-step drawing guides), with the idea being to let amateur artists feel like real artists. Using bump accelerometer as a way to easily transfer clay mesh from one device to another was also suggested. Bump accelerometer tag!

ClayAR v0.002 on iPhone 4

Thanks to SV IGDA setting a deadline of Weds the 16th for a booth demo space for nusoy, I’ve finally shoo’d away my procrastination to put in another weekend’s worth of work to release ClayAR v0.002 (Lite v0.2).

Device Endpoints

• ClayAR v0.002 for Android Qualcomm Snapdragon phones
  • Temp Beta allows for 3 different users on FM 3,4,5 each once
  • _u3 works with FM3
  • _u4 works with FM4
  • _u5 works with FM5 (default, for non-ARcam builds)
• ClayAR v0.002 for Android non-Snapdragon and Android tablets (accelerometer AR only, currently no AR cam) – fm5 only
• ClayAR v0.002 for iPhone / iPad (accelerometer AR only, currently no AR cam) – fm5 only

New features:

• “New” button actually loads a new mesh – with choice of starter mesh! (pending v0.003)
  • Single color button backgrounds (rather than with indentation)
  • Strength slider is now wider and easier to use…
  • Strength slider allows text entry
  • Info/Help overlay button now in Menu, rather than in edit/view modes
• User Login – Also has a “Just let me create now!” button, but all meshes here will be public. – v0.003 Pending
• Load Panel – Pagination! – v0.003 Pending
• Multi Mesh – v0.003 Pending
• Mirror – v0.003 Pending
• MacroMesh Operation: Stretch XYZ
• Accelerometer-based AR
• non-Snapdragon phone support, iOS support!

Lots more to come in a rather multiversioned roadmap including either multimesh or zBrush-like zSpheres (ARSpheres?) and displacement map painting

In other news, nusoy now has an AR app up in the Android marketplace — it’s just for the SVIGDA event, serving augmented reality sushi and a bowl of nusoy. It’s currently available for Snapdragon-phones only, and you’d need to point your phone at the yosunnusoy marker on my Macbook Pro to get the AR sushi to show. (For those more knowing of Ina Centaur’s legacy on SL, the sushi came from Inachi!)

But, some really neat apps made it through the contest. Here is Qualcomm’s AR Contest Compilation video.

I’m still zinging from one-too-many Cosmo’s from the #RSAC parties attending incognito under my YC alias. There’s supposed to be that “oh, no, there goes another contest,” but I just have this wow feeling at the winning apps. Paulius’ Paparazzi makes me totally ROFL – such a riot!! <3

ClayAR is briefly demo’d at 1:40 of the “Compilation video.”

And, on the future of ClayAR… Dunno. Stay tuned!