Sunil's Blog » FlarToolkit http://www.suniljohn.com/blog Sat, 30 Jan 2010 09:02:07 +0000 en hourly 1 http://wordpress.org/?v=3.0 Oh My Bosh! Design Your Very Own Chris Bosh Paper-Robo in Augmented Reality – Adding Interactivity to Your AR Apps http://www.suniljohn.com/blog/oh-my-bosh-design-your-very-own-chris-bosh-paper-robo-in-augmented-reality-adding-interactivity-to-your-ar-apps/ http://www.suniljohn.com/blog/oh-my-bosh-design-your-very-own-chris-bosh-paper-robo-in-augmented-reality-adding-interactivity-to-your-ar-apps/#comments Fri, 31 Jul 2009 14:28:25 +0000 admin http://www.suniljohn.com/blog/?p=208 bosh_lbj

With Chris Bosh’s  contract with the Raptors expiring soon, there’s been a lot of talk of him being shopped around next season. Sources say that the Cleveland Cavaliers have targeted Bosh to play along side King James.

While I don’t want to see Chris leave Toronto, it wouldn’t surprise me to see him follow the path of other marquee players before him : Stoudemire (he was big back in the day), Carter, McGrady and Camby.

Where do you think Chris will go?

With this latest flar demo you can make your predictions by redesigning his jersey in AR, download the parts to your desktop, and print/assemble your own Paper-Robo! You can also upload your own custom skin to preview what the robo will look like in AR.

For anyone who designs a killer Bosh skin through this app or even better photoshop, please send them my way and I’ll post them here! I bet some of CB4′s twitter army could make some sick custom bots.

Here’s a link to the live demo to try yourself:

Bosh Demo

Direct your iphone here (don’t waste paper, if you can help it!!):

Download Bosh Marker

Download Bosh Paper-Robo Parts

CODE SNIPPETS:

Adding mouse events to collada models and manipulating materials

This functionality is simply an extension of the helloMouse3D example that ships with the Papervision3D libraries. The only difference here is that instead of manipulating the material of a Papervision3D primitive, you are manipulating the material of a 3D collada model.

One thing I discovered is that material lists for colladas are wonky, and in order to add interactivity to 3D models  you have to add listeners directly to the to the mesh associated with that material and only when the model has completely loaded. As you can see be the live demo, the focus of the mouse on the model sometimes gets lost. I suspect that this is due to the collada model I’m using which has been to hell and back to get it rendering properly in Papervision3D. I built the model 2 years ago in Sketchup when either Papervision 1.0 or 1.5 around! From Sketchup I exported it as a collada, and brought it into 3DS max to make a uv map. I exported from 3Ds max as a collada, only to import the result into Blender, to obtain a collada with settings the Papervision3d can render.

When I have time I will try the demo with a simpler collada to see if this loss of focus is still a problem. If you have any ideas why I’m losing focus on my model please let me know!

public function addMarker(marker : FLARMarker) : void
  1.   {
  2.    // store marker
  3.    var markerList : Vector. = this.markersByPatternId[marker.patternId];
  4.    markerList.push( marker );
  5.  
  6.    var container : DisplayObject3D = new DisplayObject3D( );
  7.    robo = new Collada( FLARManager._GATEWAY + "/resources/models/myrobo4flippedblender.xml" );
  8.    robo.scale = .07;
  9.    robo.rotationX = 90;
  10.    robo.rotationZ = 90;
  11.    robo.z = robo.z – 100;
  12.    robo.addEventListener( FileLoadEvent.LOAD_COMPLETE, daeCompleteHandler );
  13.    container.addChild( robo );
  14.    this.scene3D.addChild( container );
  15.  
  16.    // associate container with corresponding marker
  17.    this.containersByMarker[marker] = container;
  18.  
  19.   }
  20.  
  21.   public function daeCompleteHandler(event : Event) : void
  22.   {
  23.  
  24.    robo.getChildByName( "Model" ).addEventListener( InteractiveScene3DEvent.OBJECT_MOVE, handleMouseMove );
  25.    robo.getChildByName( "Model" ).addEventListener( InteractiveScene3DEvent.OBJECT_PRESS, handleMouseDown );
  26.  
  27.    surface = new Sprite( );
  28.  
  29.    textureContainer = new Sprite( );
  30.  
  31.    __boshSkin = new boshSkin_mc( );
  32.  
  33.    __boshSkin.addChild( textureContainer );
  34.  
  35.    __boshSkin.addChild( surface );
  36.  
  37.    var mat : MovieMaterial = new MovieMaterial( __boshSkin );
  38.    mat.interactive = true;
  39.    mat.animated = true;
  40.    mat.allowAutoResize = false;
  41.    mat.updateBitmap( )
  42.    robo.getChildByName( "Model" ).material = mat;
  43.   }
  44.  
  45. public function handleMouseDown(e : InteractiveScene3DEvent) : void
  46.   {
  47.    //Start Drawing
  48.    if(surface != null)
  49.    {
  50.     surface.graphics.beginFill( __activeColour, 1 );
  51.     surface.graphics.drawCircle( vMouse.x, vMouse.y, 3 )
  52.     surface.graphics.endFill( );
  53.    }
  54.   }
  55.  
  56. public function handleMouseMove(e : InteractiveScene3DEvent) : void
  57.   {
  58.    if(InteractiveSceneManager.MOUSE_IS_DOWN)
  59.    {
  60.     //Continue drawing when the mouse is down
  61.     if(surface != null)
  62.     {
  63.      surface.graphics.beginFill( __activeColour, 1 );
  64.      surface.graphics.drawCircle( vMouse.x, vMouse.y, 3 )
  65.      surface.graphics.endFill( );
  66.     }
  67.    }
  68.   }

Flash Player 10 JPEG encoding

A colleague at work pointed me towards a JPEG Async Encoder class that allows you to to merge large BitmapData objects to form a large JPEG, getting around Flash player 10 BitmapData size restrictions. I didn’t need to encode something beyond 4,095 x 4,095 for this particular demo, but it’s useful to know that there is a work around.

Flash Player 10 reading and writing files locally

This demo uses a some new features of Flash 10, reading and writing files locally between the browser and OS without the need to write any server side code.

Using the FileReference.load() method you can allow a user to select a file locally and load it directly into the Flash plugin without having to upload it to a server.

FLARManager v0.5 (for FLARToolkit)

This is my first demo to include the newest FLARManager update. Eric Socolofsky has updated his framework to include some new optimizations, one of which is adaptive thresholding, which allows markers to be detectable in low light conditions.

AR Musings…

Is the wow factor is still there AR apps? Most offerings simply involve displaying a model on a glyph, which captures the user’s attention for about 2 minutes. There are a few hoops to jump through, in order to get a user to interact with an AR app: you have to print out a marker, need a webcam or have to download a specific plugin to view AR content (not as big a deal these days with broadband connections). Also imagine all the sheets of paper used around the world for a website that promotes renewable energy resources! I’m not sure what the solution is for saving our trees, other than offering digital versions of as AR marker that could displayed on a mobile device…Alternately, AR websites could offer a keyboard shortcut that mimics a marker without having to print it out.

Here are some interesting examples of interactive AR apps that have popped up recently that have started to push AR beyond displaying simple models in 3d space:

Eminem’s 3d Graffiti Art App:


Unic8′s use of  Bartek Drozdz‘s AS3Dmod library to bend primitives based on marker proximity and angle:

http://www.unic8.com/labs/flarmagnet-full-interactive-experience

The dev(s) at unic8 Studios have also done some interesting interactive AR demos with sound and light.

BMW Z4 3D paintbrush

]]> http://www.suniljohn.com/blog/oh-my-bosh-design-your-very-own-chris-bosh-paper-robo-in-augmented-reality-adding-interactivity-to-your-ar-apps/feed/ 0 Starscream, Boba Fett and Jabba the Hut…er…Jasmine the Cat…FlarToolkit AND 3D Animated Models with Keyboard Control http://www.suniljohn.com/blog/starscream-boba-fett-and-jabba-the-huterjasmine-the-catflartoolkit-and-md2-keyboard-control/ http://www.suniljohn.com/blog/starscream-boba-fett-and-jabba-the-huterjasmine-the-catflartoolkit-and-md2-keyboard-control/#comments Wed, 06 May 2009 02:03:16 +0000 admin http://www.suniljohn.com/blog/?p=156

Here’s a link to the live demo to try yourself:

Boba Demo

Direct your iphone here (don’t waste paper, if you can help it!!):

Boba – http://www.suniljohn.com/labs/boba/markers/boba.jpg

Starscream – http://www.suniljohn.com/labs/boba/markers/starscream.jpg

Print the markers here:

Boba – http://www.suniljohn.com/labs/boba/markers/boba.pdf

Starscream – http://www.suniljohn.com/labs/boba/markers/starscream.pdf

There are keyboard controls for both models. ASDW to move Starscream and directional arrows to move Boba Fett. Each character also has jump, taunt and attack controls. There are about 14 other animations I haven’t had time to program.

I re-factored the earlier multimarker flar prototype to use the FLARManager 0.3 from Eric Socolofsky at transmote.com. I was able to trace out the string names of the individual 3D channels of the MD2 models I’ve been using. I can now play the individual animations I want to play instead of looping through them all.

This snippet might help those that are trying to figure out the string names of the 3d channels of their MD2 Quake 2 models:

//_channels = new Array();
  1. //   _channels = animatedBoba.getAnimationChannels();
  2. //
  3. //   trace("channel count", _channels.length);
  4. //
  5. //   for each(var channel:AbstractChannel3D in _channels) {
  6. //    _numFrames = Math.max(_numFrames, channel.keyFrames.length);
  7. //    channel.updateToFrame(_currentFrame);
  8. //    trace("channel.name", channel.name);
  9. //   }

I also implemented Saqoosha’s hole effect using Papervision3d, in the form of trap doors, with some slight modification to what cube faces are visible. I think this has to do flipped coordinate space, when comparing a standard PV3d scene and to the flarnode. Here’s the mod:

Instead of :

var hole:Cube = this.scene.addChild(new Cube(new MaterialsList({all: cmats, bottom: cmatb}), 300, 300, 300, 1, 1, 1, Cube.ALL, Cube.TOP)) as Cube;
  1. this._green = this.scene.addChild(new Cube(new MaterialsList({all: new ColorMaterial(0x00ff00)}), 300, 300, 300, 1, 1, 1, 0, Cube.TOP)) as Cube;
  2.  
  3. Changed to:
  4.  
  5. var __hole : Cube = new Cube(new MaterialsList({all: __cubeSideTexture, bottom: __cubeBottomTexture}), 80, 80, 80, 1, 1, 1, Cube.ALL, Cube.FRONT);
  6. var __green : Cube = new Cube(new MaterialsList({all: new ColorMaterial(0x00ff00)}), 80, 80, 80, 1, 1, 1, 0, Cube.FRONT);

I went through at least 10 different asymmetrical patterns for the Starscream flar marker. Most of the time the camera would keep losing the pattern or confusing it with the Boba Fett pattern. Bad lighting and problematic marker patterns were the biggest issues.

I don’t have to use keycodes very often, but when I do it’s always a pain trying to remember what uint corresponds with what key. Here’s a snippet I picked up for using as3 to generate keycodes:

http://evolve.reintroducing.com/2008/11/21/tips-n-tricks/flash-key-codes/

The end result isn’t quite this:

starwarschess

….but each time I revisit this prototype it’s seems to be getting closer….

In the future I want to implement mac3d’s adaptive filter, to help eliminate problems with marker detection caused by poor lighting conditions.

Also what fun would spawning 3d characters  in AR be without laser canons and collision detection!?!

jasmine_jabba

]]> http://www.suniljohn.com/blog/starscream-boba-fett-and-jabba-the-huterjasmine-the-catflartoolkit-and-md2-keyboard-control/feed/ 9 FITC Toronto 2009 Top Ten Round Up http://www.suniljohn.com/blog/sunils-fitc-toronto-2009-top-ten-round-up/ http://www.suniljohn.com/blog/sunils-fitc-toronto-2009-top-ten-round-up/#comments Sat, 02 May 2009 18:18:46 +0000 admin http://www.suniljohn.com/blog/?p=135 I just got back from volunteering at FITC Toronto 2009. I HIGHLY recommend volunteering. It’s a great way to attend the sessions for free and meet amazing new/old talent in the interactive field. I’ve come away from the conference with a lot of new ideas that I can’t wait to try.

Here’s my list of things that were cool to me in no particular order:

1) Papervision X – Ralph Hauwert
- 3d renderer optimized for Flash Player 10
- Ralph used Alchemy and Pixel bender to run Quake in Flash at 60 FPS!!

2) Colin Mook – Your Phone is Your Controller
- http://www.playmegaphone.com/
- http://unionplatform.com/intro.html

3) Sourcebinder
- http://sourcebinder.org/
- drag and drop node based programming
- break out game built on the fly with audience interaction using a webcam

SourBinder Tutorial: The basics from Balazs Serenyi on Vimeo.

4) Gary Stasiuk – Visual PV3d
- Flash Visualization Application that provides a GUI interface to create and manipulate objects in 3D scenes using Papervision3D

Vizual PV3D from Juxt Interactive on Vimeo.

- http://www.juxtinteractive.com/_download/vizualpv3d.zip

5) Vector Vision
- allows you to use vectors in Papervision
- http://blog.barcinski-jeanjean.com/2008/05/16/vectorsvision-vectors-in-papervision3d/
- http://www.barcinski-jeanjean.com/entries/vectorvision/SvgExample.html

6) Golan Levin and Zach Lieberman
- Augmented Reality Magic 1.0

7) Jared Ficklin
- music visualization through physical objects
- nirvana songs visualized on paper show similarities between songs; verse – chorus – verse
- showed 4 patterns and asked which one was likely a nirvana song, the one most similar was actually a foo fighters song!

8) Mario Klingemann
- bitmap manipulation turning ordinary bitmaps into Van Gogh like paintings.
- http://www.aviary.com
- using Keith Peters minimal components

9) Pixel Bender Game
- http://www.kongregate.com/games/GlaielGamer/closure

10) The Spark Project – Japanese Flash Community
- http://www.libspark.org/wiki/WikiStart/en
Photorealistic AR

Me too (doing some AR stuff)! from Anatoly Zenkov on Vimeo.

Frocessing
- Processing like AS3 drawing library
- http://www.libspark.org/wiki/nutsu/Frocessing/en

BetweenAS3
- new tween engine that performs better than the major players tweener, tween max…
- http://www.libspark.org/svn/as3/BetweenAS3/trunk

SWFWheel
- support for mac scroll wheel no javascript required
- http://www.libspark.org/wiki/SWFWheel/en

]]> http://www.suniljohn.com/blog/sunils-fitc-toronto-2009-top-ten-round-up/feed/ 1 Revisiting Multiple Pattern Recognition Using the FLARMultiMarkerDetector and an iPhone http://www.suniljohn.com/blog/multiple-pattern-recognition-redux-using-flarmultimarkerdetector-and-iphone-flartoolkitflash-augmented-reality/ http://www.suniljohn.com/blog/multiple-pattern-recognition-redux-using-flarmultimarkerdetector-and-iphone-flartoolkitflash-augmented-reality/#comments Sun, 08 Mar 2009 02:19:58 +0000 admin http://www.suniljohn.com/blog/?p=118 I was able to revisit the spaghetti code on my FLARMultiMarkerDetector prototype and improve performance 100 fold. Thanks to the guys at Squidder.com for sharing their source for the virtual drum kit.

Here’s a post explaining their demo.

Here’s a link to their source.

The MD2 models I’ve been using should be familiar by now and are from Polycount.

I was writing this app on the train and didn’t have a printer to print the patterns. Now I have them on my iPhone, so I can test my Flar apps where ever I happen to be.



One thing I’ve noticed, is that if you cutting the patterns out or are displaying them on an iPhone it seems you have to leave a white margin in addition to the black border for the pattern to be recognized.

]]> http://www.suniljohn.com/blog/multiple-pattern-recognition-redux-using-flarmultimarkerdetector-and-iphone-flartoolkitflash-augmented-reality/feed/ 3 My Cat and Multiple Pattern Recognition Using FLARMultiMarkerDetector – FlarToolKit/Flash Augmented Reality http://www.suniljohn.com/blog/my-cat-jasmine-and-multiple-pattern-recognition-using-flarmultimarkerdetector-flartoolkitflash-augmented-reality/ http://www.suniljohn.com/blog/my-cat-jasmine-and-multiple-pattern-recognition-using-flarmultimarkerdetector-flartoolkitflash-augmented-reality/#comments Mon, 02 Mar 2009 03:48:05 +0000 admin http://www.suniljohn.com/blog/?p=101 I’ve been experimenting with how Flash and FlarToolKit handles multiple patterns using FLARMultiMarkerDetector .  I’ve found that a lot of light is needed for the detector to work reliably with mutiple markers. Here are a couple screens of my progress so far using animated MD2 models of Boba Fett and Starscream. Thanks to my cat Jasmine for the moral support.

multi4
multi1
multi2
multi3

]]> http://www.suniljohn.com/blog/my-cat-jasmine-and-multiple-pattern-recognition-using-flarmultimarkerdetector-flartoolkitflash-augmented-reality/feed/ 7