Blender: Overview of Object Tracking in Tomato Branch

by Sebastian Koenig

In this overview of some upcoming features in Blender, we show you how to use the new Object Tracking available in the Tomato branch. Object Tracking, similar to Camera Tracking, allows you to track the position of an object relevant to the camera so that you can replace objects in real footage with a 3D model.

Note: this feature is not yet available in the official release of Blender 2.60. You can download a development version from http://graphicall.org if you wish to try it.

The footage used in this video is available from here: http://www.hollywoodcamerawork.us/trackingplates.html

GSOC 2011 Tomato Tracker<

You must be logged in to upload images. Register

Discussion

77 Responses to “Blender: Overview of Object Tracking in Tomato Branch”
  1. bob anymouse
    Posts: 40

    First!

    #
    1
     Dec 7, 2011 at 9:16 am
    • bob anymouse
      Posts: 40

      Wow, that actually worked? Interesting stuff… the overview I mean.

      #
      1.1
       Dec 7, 2011 at 9:17 am
    • Caroline
      Posts: 24

      Congrats ….

      #
      1.2
       Dec 7, 2011 at 10:30 am
  2. Stefan Misic
    Posts: 4

    Wow,really amazing. And that girl from the video is beautiful too. ^^

    #
    2
     Dec 7, 2011 at 9:30 am
  3. Caroline
    Posts: 24

    Brilliant! I wondered where to get some practice footage from … I’m a really, REALLY bad camera woman :(

    Thank you, Sebastian! :)

    #
    3
     Dec 7, 2011 at 10:22 am
  4. bishop knight
    Posts: 82

    The whole time I watched this I kept picturing Alan Tudyk as “Dutch” from Transformers 3 doing the instructing. It is very exciting seeing Blender develop like this. It’s becoming more and more robust. I’m guessing this feature is the result of the current Blender movie project being developed. Time to donate to the Blender Foundation.

    #
    4
     Dec 7, 2011 at 12:13 pm
  5. comeinandburn
    Posts: 62

    @Cattitude63 this footage and a lot more comes from hollwoodcamerawork.us site in the downloads section… it’s an awesome resource and the footage is free.

    #
    5
     Dec 7, 2011 at 12:17 pm
    • Gary Parkin
      Posts: 109

      Thanks @comeinandburn. I didn’t know that. My camera shoots in HD if anyone would like some footage. I live in NJ, across from woods. LOL, everyone in NJ has woods.

      #
      5.1
       Dec 8, 2011 at 6:48 am
      • Gary Parkin
        Posts: 109

        I forgot to say thank you ahead of time, for the info and to Sebastian for the tutorial.

        #
        5.1.1
         Dec 8, 2011 at 6:49 am
  6. SeriousM
    Posts: 6

    very nice and detailed tutorial, thanks sebastian!

    #
    6
     Dec 7, 2011 at 2:46 pm
  7. kenneth plumley
    Posts: 11

    thanks for the new info. the camera tracker has become my new favorite toy at the moment. and i would love to see all these features make it into the official release. once again thank you for all the info and have an awesome sweet day.

    #
    7
     Dec 7, 2011 at 5:56 pm
  8. Jeremy Deighan
    Posts: 263

    Amazing work! Way too beyond my abilities but I respect those that can do camera tracking and those that can program the tracking software! :D

    Cheers!

    Jeremy Deighan

    #
    8
     Dec 7, 2011 at 7:31 pm
  9. Adam Falldorf
    Posts: 10

    Gotta love the fancy prop design, I have remember getting that lightsaber shortly after episode 1 was released. :P
    Great tutorial btw.

    #
    9
     Dec 7, 2011 at 8:59 pm
  10. Payton
    Posts: 12

    I love you sebastian!

    #
    10
     Dec 7, 2011 at 9:01 pm
  11. Ehab Charek
    Posts: 9

    this is awesome!!! :D
    I have one question though, does the “object’ feature work with body or facial tracking, like the movies avatar or tron?
    once again great job guys, this is going to be amazing I can’t wait C:

    #
    11
     Dec 7, 2011 at 9:04 pm
  12. Vincent Spiller
    Posts: 5

    Hey, i love the video i have tried to do this and fail but now that there’s a good tut i have done a really nice job but i have a question. is there a chance you could show how to use green screens and how to add objs without the balls.

    #
    12
     Dec 7, 2011 at 11:12 pm
  13. alwadhani
    Posts: 14

    man you are my Hero ;) , i was planning on making a short movie which require some object tracking and i thought that it wont work with blender BUT now you have made me the happiest guy in the world. i realy appriciate your helpand please keep it up.
    Hani :)

    #
    13
     Dec 7, 2011 at 11:42 pm
    • alwadhani
      Posts: 14

      i just have a question will this work for facial tracking using a steady cam in tripod?, also i dont seem to find the object menu in top of track menu??!!

      #
      13.1
       Dec 8, 2011 at 3:40 am
      • Payton
        Posts: 12

        You probably dont have the right build.

        #
        13.1.1
         Dec 8, 2011 at 11:01 am
      • Samuel Telles
        Posts: 2

        I have the relesed version, and i dont have the “object” menu… i’t a BIG BIG BIG error by blender developers

        #
        13.1.2
         Dec 15, 2011 at 5:58 pm
      • Jonathan Williamson
        Posts: 2952

        Object Tracking is not yet done for release; it should be in the 2.62. It’s better to wait releasing it than to release a buggy version now.

        -Jonathan

        #
         Dec 15, 2011 at 6:47 pm
    • alwadhani
      Posts: 14

      spot on, i just downloaded the new one ;)

      #
      13.2
       Dec 9, 2011 at 2:56 am
      • Samuel Telles
        Posts: 2

        The final release dosnt have it. Somebody can explain to me WHY dont? Its the most necessary feature for it… and somebody very smart decide not improve the Object menu.. WHY F GOD?

        #
        13.2.1
         Dec 15, 2011 at 6:13 pm
  14. kr8bit
    Posts: 1

    hey,
    just curious what are your system specs when you played around with the video and tomato tracker. My computer crashes alot and runs out of resources when I follow this tutorial. Thanks

    #
    14
     Dec 8, 2011 at 1:01 am
  15. Erik
    Posts: 1

    Hi Sebastian,

    great work as always.
    A shame the branch wasn`t available in our course at LSoD..

    Frohes Schaffen!

    #
    15
     Dec 8, 2011 at 1:16 am
  16. Chris
    Posts: 3

    I was too distracted by the woman to follow the video properly :D

    Just kidding, great tutorial!

    #
    16
     Dec 8, 2011 at 5:25 am
  17. Dusty
    Posts: 2

    This is amazing stuff!!!!

    Just out of interest, will this ever work with multiple camera angles.

    Say we are trying to get some mocap on a person, you could use 2 or even 3 cameras to keep the markers in site. Can you then use those videos to triangulate tracking?

    Or is this something that would be on the drawing board right now?

    #
    17
     Dec 8, 2011 at 6:31 am
  18. J C Roberts
    Posts: 12

    Just curious, the intro text above makes it seem like this is still only available in the graphicall “Tomato” builds, but I thought it was also part of the test release you can get through the Blender.org site. It doesn’t link to graphicall for this version, and better still, it works. That’s more than I can say for every single build I’ve attempted to try from graphicall. Each one crashes instanly and without fail every time. Since no one’s been able to even explain why, I don’t bother with them anymore. With only so much time to work on things at night, I have little patience with wasting my time generating endless “unexpectedly quit” messages on my screen instead of actually working on things.

    The one I got through Blender.org has been fine so far, but I’ve only looked into Cycles with it up to now. They say the tracker is part of it too, so why not direct people to version that works? Have there been additions and fixes since that one went up, so it’s no longer the most up to date version of it?

    #
    18
     Dec 8, 2011 at 11:49 am
    • Jonathan Williamson
      Posts: 2952

      The camera tracker is included in the Blender 2.61 testing builds, but the object tracking portion of the motion tracker is not included yet.

      #
      18.1
       Dec 8, 2011 at 12:08 pm
  19. ben
    Posts: 2

    Great tut and what a great feature set for blender. Is it possible to track an object based in a fixed camera position? ie the camera is not moving but the objects are.

    #
    19
     Dec 9, 2011 at 6:28 am
  20. Gary Parkin
    Posts: 109

    This was really good and I have a much better idea, but where and how do I get the Focal length from the camera? Even if I’m using my own HD camera?
    I’d love to see more of these, maybe in depth as to how to use the solver and what the errors mean.
    And yes, I did watch the camera tracking video,s. :)

    Any links would be really great. I’ll have to mess with this over the weekend.

    #
    20
     Dec 9, 2011 at 7:45 am
    • Sebastian Koenig
      Posts: 4

      Well, if you do the filming yourself, that should be pretty easy. You can find out the sensor-size of your camera either by camera manual or so or by googling the camera model. The focal length should be noted on the lens. But if you use the refine-tools of the solver, and first just roughly guess the focal length, you have a good chance that blender can figure it out on itself.
      The error is the average value how much the actual 3d point differs from the marker position after the solution, measured in pixels.

      #
      20.1
       Dec 10, 2011 at 5:35 am
      • Gary Parkin
        Posts: 109

        Thanks Sebastian
        It’s the actual sensor size? Ok, that makes sense. I shoot my HD with a Canon HF10. I did a lot of research and it was the best consumer camera for the money when I bought it.
        I’ll have to shoot some footage this week and play with this. It would be fun just to add a 3d object into the footage.
        Blender has grown into a full featured application. I can’t wait to see what they do next.
        I’d love to see chamfers fixed.(Rounded edges on corners).

        #
        20.1.1
         Dec 12, 2011 at 6:49 am
  21. Andrew Pace
    Posts: 59

    Amazing what Blender is capable of these days. Great job on the tut Sebastian. Thanks!

    #
    21
     Dec 9, 2011 at 10:07 am
  22. gvproductions
    Posts: 2

    I can’t help but think of armature tracking. I wonder if Blender is capable enough yet to do pingpong balls on joints of the black body suit then use the track to control the armature. Would be very interesting. Another application could be for facial tracks if the tracking data to armature manipulation works out.

    #
    22
     Dec 9, 2011 at 10:23 am
    • Dusty
      Posts: 2

      I would like to know this too.

      I was kinda hoping that using 2 or 3 cameras to capture motion in sync could be integrated into this. Though I don’t know how difficult that would be.

      #
      22.1
       Dec 9, 2011 at 10:38 am
      • Sebastian Koenig
        Posts: 4

        In theory it is possible, as (from what I understand) the principles are somewhat similar, but lots of new tools would be needed for that, and currently there are other priorities. I don’t see why it should not be possible, but I doubt it will be soon. :)

        #
        22.1.1
         Dec 10, 2011 at 5:37 am
    • Gary Parkin
      Posts: 109

      You could simulate it by using cubes attached to the balls like bones and see what happens. It would be like a stick man.

      #
      22.2
       Dec 12, 2011 at 6:51 am
  23. Severus
    Posts: 1

    I keep getting this error: “at least 8 tracks on both of keyframes are needed for reconstruction

    #
    23
     Dec 11, 2011 at 1:04 pm
    • dewington
      Posts: 1

      Try different settings keyframes for A and B.

      #
      23.1
       Dec 14, 2011 at 5:06 am
      • Patrickmac110
        Posts: 1

        im having the same problem, what do you mean by “Try different settings keyframes for A and B.”

        #
        23.1.1
         Jan 31, 2012 at 5:12 pm
  24. lkjhf
    Posts: 3

    wow… my computer blocked this for pornagraphy…

    #
    24
     Dec 14, 2011 at 3:36 pm
  25. Makaco Studios
    Posts: 4

    look at this has effects made in blender: http://www.youtube.com/watch?v=zbYR2DWCE8c&feature=player_embedded#!

    #
    25
     Dec 23, 2011 at 2:55 pm
  26. smilebags
    Posts: 1

    Hey! I’m running Blender 2.61, and I can’t seem to get the “Objects” drop-down in the properties bar. I know it’s just something simple I missed, but can someone please give me a hand?

    #
    26
     Dec 24, 2011 at 1:50 am
  27. Chris
    Posts: 1

    What is the hotkey for adding a marker? I’m getting tired of hitting “Add Marker” every time :)

    #
    27
     Dec 24, 2011 at 1:56 am
  28. Chabane
    Posts: 5

    thanck for all, but please put your video on youtube ,thanck you mr Sebastian Koenig

    #
    28
     Dec 25, 2011 at 12:48 pm
    • Gary Parkin
      Posts: 109

      @Chabane. Why not join Vimeo? It’s free. Or if you’re using Firefox, you can get a plug-in to save any video content and watch it later.

      #
      28.1
       Dec 27, 2011 at 6:04 am
  29. Chabane
    Posts: 5

    thanck you gary , I ask that because the big video on vimeo always stop at 95% or 98%, for that I ask the autor too put on youtube only fir that

    #
    29
     Dec 28, 2011 at 1:47 pm
  30. Physics Guy
    Posts: 13

    Hi Sebastian,

    I don’t know if you still read these comments, but I have a question considering the camera track you (or Sergey) made. In the 3D view (at time code 12:29 for instance)
    we can see that the marker is not at the corner of the window, where it is in the
    tracking window. I have similar offsets in my track.

    This would be a bad thing if one wanted to use the markers to construct geometry.
    How should I deal with this. I would greatly appreciate your input!

    #
    30
     Jan 5, 2012 at 3:50 pm
  31. Matt Clegg
    Posts: 1

    I have Blender 2.81 and I do not have the OBJECTS tab on the right hand side. Is it not out yet or do I have to enable that tab somewhere?

    #
    31
     Jan 7, 2012 at 7:00 am
    • Jonathan Williamson
      Posts: 2952

      Hi Matt,

      Object Tracking is not included in Blender 2.61. It should be added for 2.62, which is likely to be released in the first two weeks of February.

      -Jonathan

      #
      31.1
       Jan 7, 2012 at 9:35 pm
    • PhysicsGuy
      Posts: 2

      If you go to the Blender website, you can download the latest nightly build from svn.
      This includes object tracking. It is not an offcial release, so it might not be as stable as 2.61

      #
      31.2
       Jan 8, 2012 at 12:12 am
  32. Paolo Di Luca
    Posts: 2

    Nice tutorial,
    but somone can tell me how i create a object tracking with a static camera?

    like this video for example :http://vimeo.com/34442645

    thanks :D

    #
    32
     Jan 13, 2012 at 11:23 am
    • eamon driscoll
      Posts: 13

      I just let the tracker solve a bunch of “still points” in the footage an it worked pretty well for me. as long as those points are not obstructed by the objects that move in the frame.

      #
      32.1
       May 11, 2012 at 1:05 am
  33. Paolo Di Luca
    Posts: 2

    or this one

    http://vimeo.com/33223317

    #
    33
     Jan 13, 2012 at 11:25 am
  34. rangiz
    Posts: 2

    I have the same question like Paolo Di Luca, can anyone explain it?

    #
    34
     Feb 2, 2012 at 4:20 am
  35. mathieu
    Posts: 5

    I got the same question: And what if the camera is static?

    #
    35
     Feb 15, 2012 at 9:34 am
  36. Gustav Nilsson
    Posts: 13

    Same question here :) Would really like to know how to do it with a static camera.Do one really have to add 8 trackers and solving a still camera to do it? That is the only way I have been able to do it but I wish there was an easier way.

    #
    36
     Feb 18, 2012 at 8:22 am
  37. Gary Parkin
    Posts: 109

    There’s an excellent mini part of the course on hollywoodcamerawork.us
    Watch: HOW TO PLACE TRACKING MARKERS
    http://www.hollywoodcamerawork.us/vfx_sampleclips.html

    I hope this feature stays in Blender.

    #
    37
     Mar 9, 2012 at 11:50 am
  38. cotric
    Posts: 1

    Thanks for this nice tutorial.
    But I don’t know what can do with this camera tracking…
    Anyone teach me please, detailed.

    #
    38
     Mar 10, 2012 at 1:24 am
  39. Jeremy Deighan
    Posts: 263

    Whenever I solve for the Object Motion, I don’t get the empties in my viewport. Nothing seems to work correctly from that point onward.

    I tried to still use an Object Solver Constraint and everything starts screwing up. Can anyone help out?

    Jeremy

    #
    39
     Mar 14, 2012 at 12:18 am
    • Jeremy Deighan
      Posts: 263

      Ok Sebastian emailed me to let me know what is going on. In Reconstruction, the Scale gets set to default at 0.00001 and needs to be changed to 1. Thanks Sebastian!

      #
      39.1
       Mar 15, 2012 at 10:53 pm
  40. Jeremy Deighan
    Posts: 263

    At 7:33 you say you use “ALT + RMB” to manually track the markers.

    I was using the key G (grab) and moving with LMB.

    Is there a difference?

    ALT + RMB doesn’t seem to do anything for me…

    #
    40
     Mar 15, 2012 at 1:37 am
  41. Aaron
    Posts: 8

    Does object tracking still work with the camera locked down on a tripod?

    #
    41
     Apr 20, 2012 at 11:42 am
  42. eamon driscoll
    Posts: 13

    Hi, Im making an video of a person with a monkey on their back. I followed some tuts on blender cookie and have gotten a camera to track before. So then I tried to just make a “locked off” camera and just track an object. To keep it simple.
    Well it worked somewhat well. Here is the result of my first/only test. You might see my problem in this video.

    https://vimeo.com/41963729

    So now I’m trying to do this for real and I have an issue.
    I can never seem to get my error ratio down to 0. So I have a scene that has 1.65 (object error) which is pretty good (for me). But after I add the object solver constraint to my root bone it looks good for like 366 frames then on frame 367 my character moves in an odd way. but goes back to being good for maybe 10-15 frames then will go into another crazy position on like frames 436-440 then on 441 all is well again.

    I know this is because of my 1.65 error ratio but when i view the graph (in the movie clip editor) of when the track goes to hell, I see that I only have a very limited amount of balls showing on the footage (meaning I can’t solve that few frames). So I would like to somehow convert the position-rotation-scale that’s affecting the object due to the object solver constraint into key-frames. Is this possible? Cause then couldn’t I just erase the keyframes on my wackey track and let the program just do “tween” animation for those?

    #
    42
     May 11, 2012 at 1:02 am
  43. Chris Ferguson
    Posts: 66

    Could you do another tutorial on this? I’m having issues with getting my plane versus camera position to look right. For instance, I did short video of my yard and the plain goes vertical from the grass.

    Is it possible to utilize camera tracking to create some animations for use on a character?

    #
    43
     Jul 19, 2012 at 7:19 am
  44. de
    Posts: 3

    good job but when i camera track how do i increase the amount of memory cache allowed

    #
    44
     Jul 28, 2012 at 8:47 pm
  45. jamais vaincu
    Posts: 1

    Hi there, I think I followed every step of your awesome tutorial the most closely I could but I’m stumbling on a stingy problem, by the time I did all my tracks, I can’t click on the “setup tracking scene” button…
    Why is that so ?

    #
    45
     Oct 17, 2012 at 4:41 pm
  46. jussi latvala
    Posts: 1

    Miksi ei voi näyttää näissä tutoriaaleissa edes jonkinlaista valmiiksi saatua videopätkää malliksi. Aivan typeriä tutoriaaleja.

    #
    46
     Nov 7, 2012 at 6:19 am
  47. howarto
    Posts: 2

    Mmm… is amazing but… How I render the image? I try with nodes but the image is with zoom :S

    #
    47
     Dec 16, 2012 at 4:59 pm
  48. troiano27
    Posts: 3

    In camera tracking, will the more objects tracked give a better camera solution? I know I was to get my solve error less then .5. It asks for a min of 8 points to be tracked, is more better?

    #
    48
     Dec 31, 2012 at 1:14 pm
    • Alex Telford
      Posts: 454

      More is better, but often less is more as you don’t want lots of markers to mask out, and low quality points will bring down the overall track quality.
      -Alex

      #
      48.1
       Dec 31, 2012 at 4:02 pm
  49. Nico
    Posts: 14

    After solving the object motion, why do the markers for the object end up inside the camera, and very small?
    thanks,
    nico

    #
    49
     Apr 2, 2013 at 5:17 pm
  50. Marius Giurgi
    Posts: 7

    I have a suggestion to make to Sebastian :) I’m not sure if it can actually help the tracking algorithm, but I was thinking, would it be possible, let’s say with the object tracking, once certain markers are solved and a rigid 3D structure for those markers is accurately established, to help the occluded markers be predicted in case they happen to jump around due to occlusion. Basically, say you have a marker that occasionally gets occluded or is blurred or for any reason has a poor solving error. Instead of the bad solution for that marker, the algorithm (using the predetermined 3D structure that includes that marker) uses this information to better estimate where the marker should be, therefore sparing the user having to reposition the misplaced marker.

    #
    50
     May 3, 2013 at 6:02 pm

Leave a Comment

You must be logged in to post a comment.