Sunday 16 September 2012

Converting pictures into a 3D mesh with PPT, MeshLab and Blender



Obs.: Please, read the article http://arc-team-open-research.blogspot.com.br/2012/09/extreme-sfm-fast-data-acquisition-and.html with technical information that is important and complementary about the technique, place and manner that the photographs were taken.

SfM is a powerful technology that allow us convert a picture sequence in a points cloud.

MeshLab is a usefull software in 3D scanning tool, that is in constant development and can be used to reconstruct a points cloud in a 3D mesh.

Blender is a most popular opensource software of modeling and animation , with a intuitive UV mapping process.

When we joint the three software they allow us create a complete solution of picture scanning.


The process will be described superficially for that already have a few knowledge about the tools used to do this reconstruction.



First of all, was needed a group of pictures that was converted in a points cloud with Python Photogrammetry Tools.

The picture was taken without flash. This make the process harder in the future, when is needed use the image how reference to create the relief of the surface.

MeshLab was used to convert the points cloud into a 3D mesh with Poison reconstruction.

The surface was painted with vertex color.

The 3d mesh and the points cloud was imported in Blender.

The points cloud was imported because it have the information about the cameras point (orange points).

Using this points was possible placed the camera in the right position.

The vanishing points was matched using the focla distance of the camera. But, how we can see in the image above the mesh didnt match with the reference picture.

To point the camera was needed to orbit it manually.

Blender have a good group of UV Mapping tools. It is possible to use only the interest region of the picture to make a final texture map, how we can see in the infographic above.

So, in this process each viewpoint texture was projected using a picture. Above we can see in right the original image, and in the left the mesh with the projected texture. This appears to be perfect because the viewpoint of the camera is the same viewpoint of the picture.

But, if the 3D scene is orbited, we can attest that the projection works well only in the last viewpoint.

So, a good way to make the final texture is using the viewpoint of the picture to paint only the interest area.

When the scene is orbited we can attest that only the interest area was painted.

The surface have to be painted using some viewpoints, to complete bit by bit the entire texture.

We can see the finished process above. It isn't needed using all pictures taken to build the final texture. Depending on complexity of the model inly four images will be needed to complete the entire texture.

Now we can compare the texture process and the vertex paint process. In this case the texture process was more interesting to be used.

The resulted mesh have a hight level of details and nevertheless can be used to be viewed in realtime (see the video in the top).

To increase the mesh quality, we can use the Displacement Modifier in Blender. It project the relief of the surface using the texture how reference.
The final result:








CREDITS:
This article was possible thanks to the kindness of Dott.ssa Paola Matossi L'Orsa and Dott.ssa Sara Caramello and with the permission from the "Fondazione Museo delle Antichità Egizie di Torino".

9 comments:

  1. Congratulations! I always enjoy your articles.

    ReplyDelete
  2. Great ! Only photos and Open Source software.

    :)

    The future of 3D Modeling is here.


    BobMaX

    ReplyDelete
  3. o.O
    it make me be scared! look at these awesome results... oh my!

    ReplyDelete
  4. Blender don't have a native support to import Bundler scenes. You need some kind of plugin/script like:

    http://projects.blender.org/tracker/index.php?func=detail&aid=30893&group_id=153&atid=467

    rgaidao

    ReplyDelete
  5. Mr. P tutorial with the process done with Meshlab (more to come, using bundler output):

    http://www.youtube.com/watch?v=Pv6_qFIr7gs

    rgaidao

    ReplyDelete
    Replies
    1. thank you Ricardo,
      Matteo's tutorials are always interesting :)

      Delete
  6. thanks for sharing. Would it be possible to have a video tutorial of your work?

    ReplyDelete
    Replies
    1. Hi Luciano!

      I dont have an especific tutorial of this work, but have two that you can use to do it.

      You can see this to reconstruct on object: http://vai.uibk.ac.at/dadp/doku.php?id=ppt_loewen_en

      And this to project the texture on the object: https://www.youtube.com/watch?v=liWluJimWM0 (I used it for my first paint)

      Thank you!

      Delete
  7. Hi Luciano,
    waiting that Cicero will answer your question regarding Blender, I can tell you that there is a way to do more or less the same thing with MeshLab (obviously if you did the model with SfM/IBM techniques). The video Ricardo posted in the previous comment is done by Matteo Delle Piane and shows the process (http://www.youtube.com/watch?v=Pv6_qFIr7gs). Matteo also wrote me that he is working on a better integration with MeshLab and various SfM/IBM software.
    Ciao.

    ReplyDelete

BlogItalia - La directory italiana dei blog Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.