This year’s Google Summer of Code brings some exciting new features.
One of them is Camera Tracking, which is currently being developed in the so called “Tomato Branch” by Sergey Sharybin and the guys from LIBMV, which provides the algorithms for the tracker. In this video tutorial you will get an introduction how to do supervised manual tracking and scene reconstruction of some HD video footage. But the development is so incredibly fast that by the time you see this tutorial there are probably already new features and it will be even easier to use the tracker.
If you want to make sure to keep up with development, it will be best to build the tomato branch yourself. See the links below for more information.
Or you use the easy way and download from www.graphicall.org.
Building Blender in OSX: http://www.blendercookie.com/2010/10/25/building-blender-in-osx/
Building Branches on OSX: http://www.vimeo.com/25567856
The latest tomato builds: http://graphicall.org/?keywords=tomato
Update: In the short time since the above tutorial was published there are already some good new features!
first
great thxs
You sure like doing tracking in Blender, Sebastian? I’ve saw your other videos too. Impressice work and thanks for this video on BlenderCookie!
Keep up the great work!
I think tracking has become Sebastians favorite new toy!
-Jonathan
So true.
Especially because Sergey provides me with cool new features on almost an hourly basis. The 3d tracker is so stable, fast, accurate and awesome already, and it doesn’t even exist a week yet! (though to be fair, libmv has been working on the algorithms since 5 years… so kudos to them!)
u know when u miss a track or it stopps in the middle u can uncheck Disabled in the top right and manually place it for a couple of frames. It works for me
hey Sebastian! great to have you doing some videos for Blender Cookie. I followed your tutorials on other sites and asked for tutorials by you on Blender Cookie about Compositing,VSE etc… and … Camera tracking, which I wanted to learn.
I download this one and as a Citizen I hope you’ll post many more here
Thanks!
Thanks BC and Sebastian. Very cool.
Cool, solid tutorial. More of this, please. Could you do a tutorial were you do the next step with adding some textured object with proper lighting, maybe some movement or some effect? Does this already track moving object, like if that cube was moving?
Unfortunately, object tracking is not supported yet.
You are “only” able to track the camera movement itself. This means, you can place a cube in a static scene. You cannot attach an object to a moving, real filmed object (e.g. a person).
If this is also on the tomatoes branch todo-list, than that’s great news
@Sebastian: Thank you for your commitment to the blender community. Our interview from the German BlenderDay will take a few weeks though.
Regards,
BlenderEi
not realy true
nnhttp://vimeo.com/25270443
Hay, thanks for this vid. Its great. THX
I do not have the movie clip editor in 2.58a, did I do something wrong or is it this version of blender?
You have to use the Tomato branch build from Graphicall.org
This is a branch.
The trunk is the official version of blender.
Branches are made by programmers when they are coding new features, and only when these new features are finished, they are merged into trunk.
Currently tracking is not a finished feature, and it’s in the tomato branch (GSOC 2011 student branches are called in vegetables)
Personally, I don’t think such official tutorials should be done until the feature is finalised, as I bet a lot will change when it’s done, and these tutorials will have to be redone…
I am pretty sure these tutorials would be re-done, don’t you think?
That’s what makes Blender Cookie great, imagine how difficult it would be if there were no tutorials about Camera Tracking with the Tomato Branch. Sure you could probably figure it out, but there are people that certainly would find the tutorial useful.
Personally, I think people should write tutorials on anything because everyone needs some information for just about anything.
It doesn’t hurt to write the tutorial again, just like it doesn’t hurt to provide free education.
Tomatoes are in fact fruits, not vegetables.
very cool
thanks for making the tutorial
Now these are the tuts i like
Very nice tutorial
Thanks you… Very helpfully…
Mine isn’t working. The camera’s jumping around alot.
Thanks Sebastian for showing the current state of 3D tracking in this great tutorial!
How many traking points are enough?
Sebastian, great one there!
Just one thing, when you are compositing, all that stuff with the alpha, the extra mix node and such, is not needed, you just link the alpha of the render layer to the factor of the mix node and that will exclude the outer part.
In theory that is correct, but it is not very practical, because then I have no control over the factor. Not always you want to multiply with 100% over the footage, especially not when you are multiplying the shadow pass. With this method I can nicely control how much of the AO or shadow I want to use.
You could always use a math node set at multiply to control it, but it all comes down to workflow, a person uses whatever feels comfortable and works
Cheers!
Sebastian, thanks so much for keeping us up to date on this. I’ve been hoping to experiment with some ‘Real/CG’ integration this year, seems maybe Tomato could be showing the way now
The camera sensor of couse, that was my problem…
Very cool tutorial. Thanks Sebastian
Finally! A tracker that works with 2.5! Voodoo only works with 2.49 for me, so I can’t do my vfx explosions xD
Really great tutorial. There were some things I couldn’t figure out from your
timelapse video on blendernation. Your tutorial here really cleared things up.
Thanks!
Okay. Wow!
There is so much potential with this. I’ll definately be messing around with this.
In fact, I already know of a project I’m hoping to do in the near future where this will be awesome and invaluable.
Excellent work Sergey!
Thank you for the insight Sebastian!
Wow. Just watch the update. The tools to get the orientation right look really
useful. Also, the mesh reconstruction using the bundles is crazy cool! This basically means one has a whole image based modelling and rendering toolchain inside blender! Thanks again for the tutorial and thanks to Sergei for the great work!
Amazing tutorial Sebastian. Love the tracking looks like it works great and perfect timing for me.
One question in the first video you say that you already made a tutorial about lighting an object for better integration on another site. What was the websites name could not understand?
Thanks a lot Cody!
That other tutorial is over at cmiVFX.com http://cmivfx.com/tutorials/view/255/Blender+3D+Compositing
(Sorry for advertising for the competition here, Blendercookie
It covers everything you need to know for 3d integration into live action footage.
For a tutorial as awesome as that one I think we’ll let it slide
-Jonathan
Hey seb (can i call you seb? i’ll do it anyway
) great tutorial 
))
Just wondering if the tutorial at cmiVFX you are talking about is one that I have to pay for. (have looked at the side some times, but haven’t been able to figure out their system. (also it seems a little expensive for a tutorial
Great Tutorial!!! nnI have just problems with the Sensor Widht.
Sebastian,
In the newer builds, the sensor under “Camera Data” is now two numbers (x and y), as before it was only the width. What would be with height for the footage in the tutorial?
I keep going through this, but my camera only rotates around it’s origin (doesn’t move at all on X/Y/Z, just rotates). I’m thinking it’s because I’m using newer build that now requires width and height of the sensor to solve correctly. Any hints on this?
Great tutorial by the way!
I think I can help.
So what I have found is very useful is to first, look up your “sensor width” on the specs website for your camera.
I do not have a DSLR; I have a Canon sx230 HS. So I have a pretty small sensor. After looking it up on the Canon website I saw that I have a type 1/2.3 inch sensor. It is important that you understand this is a “type” and not an actual dimension of the sensor. For the width and height dimensions in millimeters, I went to this table on a Wikipedia page:
http://en.wikipedia.org/wiki/Image_sensor_format#Table_of_sensor_sizes
After finding my sensor type, I can see that the 1/2.3″ sensor has a width of 6.16 mm and a height of 4.62 mm. This is the key!
After using this information, rather than an informed guess, my camera solve improved drastically. Now it is very easy to get a near-perfect track (keeping in mind the keyframes for initializing the solve).
Hope this helps!
Hi Great tutorial
I have tryed to do some tracking myself, but I can’t make it work. When I track a marker with good contrasts it only tracks for a few frames, is there any way I can fix this?
Usually that shouldn’t happen. Make sure the pattern area is 11px and search area 161px. Tracks rock-solid here. And if it stops tracking, as encn said, press “d” to enable it again and bring it back on track. Hint: For manual frame by frame tracking select marker and use alt+left/right arrow.
re: “u know when u miss a track or it stopps in the middle u can uncheck Disabled in the top right and manually place it for a couple of frames. It works for me”nnMoving a tracking point with the G button also automatically re-enables it
Awesome. Can’t wait to have time to play with this.
WHHOOOOOOOOOOHOOOOOOOOOOOOO! INTERNAL BLENDER TRACKING! Now i can spend more of my money on blender cookie instead of buying tracking softwear! : )
Thanks a lot for dedicating time to this project. Tracking is a huge benefit for blender cgi integration.
Sebastian, I dont know, how I can get the sensor witgh. I am checked allready all about my cam and about the yout link, but I cannot find something!I am really “verzweifelt”! Its just possible with CCD or also with cmos sensors? My sensortyp is 3MOS [1/4,1"]. Thank you a lot for your help!!!
Did you see my comment above? On the Wikipedia page, there is a listing for the 1/4″ sensor type (but not your 1/4.1″). You could use the width and height listed for that sensor to begin with. If you still can’t find info for your specific sensor, you could always try interpolating to see if that gets you values which seem appropriate.
Thank you!! Its already much more better!
Hey, I’m having trouble installing the build. as soon as i want to extract i get a lot of errors saying i cant open/extract the files
Sounds like your extraction software is having trouble. I would recommend trying Peazip, it’s small and reliable. http://peazip.com/
hmm, it still doesnt work, it says the file might be corrupted, but i just downloaded the 32 bit build from graphicall =(
this is one hell of an awesome feature pack!!!!
Awesome introduction and explanation of the tracking workflow, great to see that you are back with blendercookie for this one Mr. Koenig.
Always been a fan of your tutorials and I think this special subject is a must see for every blenderhead around.
Thanks for the update aswell *thumbsup
Much appreciation for your efforts, best wishes and regards.
Nixon
Thanks Sebastian for more helpful insights into how to use this great new tool! & hats off to the developer.. + Jonathan your tutorials are brilliant!! you have a great gift. Thanks for sharing =)
I’d just like to say WOW! I was playing around with manual tracking not too long ago… LOL It took a lot of work and unending patience… I’m really glad to see people working on this. It looks soooo easy! Great tut Sebastian! And thanks for the update, I really liked the little bonus on reconstructing a virtual set! This tutorial has renewed my excitement about Blender! Please keep us up on this man! And I’d like to see some tips on cleaning up Cycles images too! I need to get a newer version of Cycles, but how do you remove those last 50 or 100 fireflies from an otherwise beautiful image?????? Especially for animation…. Trying to do it by manually painting them out leaves shoddy or VERY time consuming results and is out of the question for animations. I’d love to hear more about either of these two topics. Thanks for the great content BC!!!
Hey. Looks that the 3d tracking in blender is making a lot of progress. Thanks for the tutorial but woludn’t it be great that the tracking data could be exported to other programs like After effect? Becouse as we know lot of cg artist work in many programs at once so that feature would be great. Blender wolud become an important alternative for commercial 3d tracking programs. Or maybe thats already possible in a non complcated, reasonable way?
Masluch, have you checked this out? http://www.blender.bartekskorupa.com/-video/109.html I would guess that baking from camera constraint to f-curves might be involved to make the tracking solve exportable (spacebar-search “bake action”)
yes heard about that, thanks for the info though. A thought maybe this two things could get together especially that there’s a lot of development going on right now. Mayby the developers will think of such option implemented to 3d tracking
Thanks Sebastian!
Hmm, I tried projecting my footage the same way that you did, Sebastian, but for some reason I have to set the X and Y scale for the UV Project modifier to .193 for it to match my background movie clip.
Maybe I’ve done something silly here, but could this somehow be a result of the scaling that I’ve done using the Scale operator in the Movie Clip editor?
I noticed that you did not have to scale anything in your example. I’m a bit perplexed.
Well done, of course!
I’m a noob at this stuff. So take the next question as such. For inserting object meshes into the scene and getting more accurate shadows represented, could you track the source of a shadow, like the corner of a window, and the corner of a shadow like the one on the ground representing the where the light comes through past the tracked corner of the window….*takes breath*…and then align a light source to the direction from window to shadow?
That is a pretty cool idea I have to say! Will try that. Thanks!
excuse me how i can use the magic mouse with blender
doesnt work what happens is it tacks for 48 frames then stops, then on the right the little video thing turns pink. HELP
that means the tracker is disabled because it lost track. that happens. press g to move it back on track, that should enable the tracker again. you can also press d to enable it and then move it.
wow i luv this
i managed to track successfully, but now im tryinh to get to the cool stuff. when i assign a constraint solver and click on solve camera, nothing happens.
from what i have read i may need the dimension code for my footage, am i right, but i downloaded this from internet and no idea what the details for this is.
i wish there is a default value for this or anyway i can bypass this step.
hm…
I’m having trouble caching. Blender keeps crashing after about 200 frames. I have 4 gigs of ram. Help?
What is this? I thought I was learning how to model a tomato. jk nice
Great introduction and tutorial! It allowed me to create a tree growing out of ice cubes I put into the ground: http://www.youtube.com/watch?v=cfgriG6Wqrw
Amazing! No need for syntheyes anymore!
Is anyone else having trouble with image sequences not updating when used in the movie clip editor or as background images? I’ve tried .png and .jpg sequences named test_nnn.png (or .jpg). .MOV movies work better, but only update when scrubbing in the timeline, not going frame-by-frame. I’m having trouble with both Tomato branch and the 2.58a official build.
I’m using Ubuntu 10.04 64-bit. Any help is appreciated.
-Tom
I’m not sure if this is what you’re seeing, but I did notice a bug where if you hit Alt-A with the mouse cursor within the timeline, the timeline and 3D View windows panels, but not the Movie Clip panel. Try hitting Alt-A with the mouse cursor in the Movie Clip panel.
Thanks John. Turns out it was operator error. I’m not sure how I did this, but when I separated my .MOV file to an image sequence, I generated 500 copies of frame 1. The display was updating, but all the images were identical. That’s several hours I won’t get back.
I haven’t gotten around to using this but does this look to be a syntheyes killer?
After setting up the markers and clicking “Solve Camera”, I’ve been running into issues where the camera just resets back to its original orientation during some frames, while it is positioned correctly in other frames. Tweaking the first and last keyframes seems to help, but is there a better way to guarantee a good solve?
I have the same problems as well from time to time. There are numerous factors that can have an effect on the solution: the right focal length, sensor size etc. But I would say the most important aspect is if the markers are on screen long enough. On footage with lots of movement the trackers can easily slide off or just leave the frame, and if there “life” is too short it can confuse the tracker. Often their bundles in 3d space are way off. Try to find and eliminate those and solve again. Sometimes that helps.
I’m having trouble solving the track. After I solve, all the tracking bundles look like they’re in the right place and the camera movement matches the video, but my camera is always pointed in the wrong direction. Sometimes the bundles are behind it and sometimes the camera is rotated the wrong way. If the 3d camera were rotated a bit to the left then it would be perfect. What am I doing wrong?
I see there’s now a feature to automatically detect and add good track point candidates (“Detect features”).
Thanks for the great tutorial! Is there any info out there of how to composite the 3D into the movie clip fro a beginner? Thanks again!
Thanks Sebastian!
At first the tracking wasn’t working. But finally after 5 times of re-doing it, It finally worked and im really pleased with the results
Again thank you!
Looks very cool, but i have a problem.
I am doing it like you but in console it writes “No camera for frame 0″
WTF? I dont know where is problem, i tried everything. Please help…
many thanks !
I didn’t find movie clip editor.plss!help me I need ıt.where can I find ??
Which version of Blender are you using? The Camera Tracker is only in development versions of Blender, currently, and so you will have to download one from Graphicall.org
-Jonathan
That pillar goes right in front of the cube! How would one go about making a mask so that you can simulate that happening when you render it out?
this is great..!!! tks.!!
A-M-A-Z-I-N-G! Thanks!
OMG THANK YOU! I have one problem though. When I render using Nodes, only the 3D objects render. But when I hit F12 with everything going through the File Output Node, it renders both the Footage and the Objects. I’m Doing Something Wrong. Please Help!
This doesn’t work for me… I do it exactly as he does it and my cube always slides around all over the floor… What’s weird is that it’s a good track and it lines up with a background image, but when rendered, it doesn’t line up. Help anyone?
Thanks Sebastian. This was pretty cool.
In the update you constructed a fake floor and added a uvmap to it and a camera projection. I watched it 3 times trying to get it. Can you please elaborate on this?
It looked really neat for rotoscoping, or occluding an object.