This article demonstrates the use of TBGL for combining single picture with digital "actor" rendered as 3D object.
Our goal will be to create scene where rusty UFO flies over peaceful city. The movement is marked on the storyboard shown on Pic. 1. The script will produce series of bitmaps, which will create frames of videosequence finally assembled using freeware VirtualDub tool.
Elements of the scene
TBGL module and Scene-Entity system will be used to accomplish the job. The scene consists from few basic elements:
- camera and its target pivot
- background photo
- UFO model
Camera is essential, as it represents view on the scene. In reality, the white house in the centre of the photo is approximately 200 meters away from the real camera. During the shooting it was no put on plane, but pointed up and also sideways, marking place about 10 meters above the observer on the wall of white house. The whole situation is demonstrated on Pic. 2.
To recreate the real situation in virtual TBGL space, virtual camera will be placed to origin of coordinates and will point to the pivot point (-150, 10, -150), which is the point at the wall of the house in the reality.
The pivot of the camera is used to enable one very important thing – shaking of the camera. As still image and 3D model are used, to add the sensation of watching video captured by camera in hand it is enough to shake the pivot point. This way synced shaking of background and other 3D objects added to the scene is obtained.
The background photo has 4:3 proportions. It is sad fact that older 3D cards do support only resolutions of power of two. For this reason the original image at 1024x768 should be "normalized" to power of two dimensions by adding 128 pixels to top and bottom, with final resolution of 1024x1024.
The quad with background image should be scaled in a way it is not completely visible on the screen. This is just to make possible shaking of the image. In case the image would be scaled to fit the frame, the borders of the quad would be visible during the shaking which would damage the impression from the effect.
To make sure the image will be always perpendicular to the camera, TBGL_EntitySetTarget can be used to make background entity look directly at the location of camera. To prevent interaction of light and fog with the frame, it is useful to place render code for textured quad with background between TBGL_PushStateProtect / TBGL_PopStateProtect.
The UFO model can be created using any 3D modeller with support for OBJ export. The OBJ to M15 tool is recommended for conversion of to TBGL M15 file format.
You can animate the UFO any way you like, for example using TBGL_EntityPush and TBGL_EntityTurn commands.
Simulating light and atmospheric conditions
The photo has been shot at very special light conditions: on the morning, when the front, bright face of the white house is directly illuminated by Sun, and the other side is basically not lit. It is essential to transfer these light conditions to our virtual space as well.
Light model in TBGL module has the following colour related properties:
- ambient colour
- diffuse colour
- specular colour
As the surface of the house is not shiny, we will completely ignore the specularity for this case. The ambient colour is present on faces not lit by light, while the directly lit faces are painted using sum of RGB components of diffuse and ambient light colours.
Using basically any bitmap editor, including MS Paint, it is possible to measure the bright face has colour average equivalent to approximately 208, 214, 193 and the dark side of the house has colour of 100, 126, 138.
This value can be directly set to the ambient component of the light, as listing of Code 1 shows:
TBGL_EntitySetAmbient(%sScene, %eLight, 100, 126, 138)
The final paint of the light on fully illuminated face is sum of the diffuse and ambient parts. That means, if colour of the bright wall of the house was 208, 214, 193 and the dark 100, 126, 138, then the diffuse component of the light equals their difference
TBGL_EntitySetColor(%sScene, %eLight, 208-100, 214-126, 193-138)
Last note on the lighting: we are lucky the house is white. In situation where scene does not contain white object it would be advantageous to prepare simple white cube painted with matt white colour. This way the light conditions can be again analysed the same way as for the white house.
As the UFO will be truly “alien“ element to the scene (2D background vs. 3D object), it is good to somehow improve the feeling of the depth. When you stand on high mountain, you can see the terrain in distance fades away in some kind of fog, even at clear weather. This atmospheric effect will be our key to improve the depth feel.
This can be simulated using using TBGL fog. The color of the sky around the house is ideal candidate for the fog colour as well. It can be for example 206, 224, 229. It is necessary to keep in mind the setup of fog must be done with precision, as too dense fog would make the UFO disappear very sooner than is needed.
Producing frames for video
To finally generate the sequence, it is needed to export series of bitmaps first. This can be achieved using TBGL_SaveScreenshot command. It is good idea to store the frames in separate directory and name each frame in the same way with increasing number. This saves us the task to manually arrange the frames later.
When producing video, it is good to keep the quality as high as possible. To avoid production of teared frames, it is wise to enable vertical synchronization using TBGL_UseVsync. This ensures the screen will be updated only when whole frame is ready.
Another very important note is to use manually specified constant frame rate when using the script in frame generation process. Using functions dependant on time, such as timer, getTickCount or hiResTimer_Get, would be bad choice depending on actual PC performance(slowed down by saving bitmaps), resulting in irregular playback of the final video.
Post processing in VirtualDub
Once the numbered frames of animation are generated, start VirtualDub and drag the first file in series to the program window. The rest of files will be added automagically thanks to fact they are numbered in ascending order.
Then it is necessary to select target frame rate using Video/FrameRate option, ideally the same as used when rendering the frames. It is also recommended to add suitable post processing filters. For this case it is recommended adding the motion blur, interpolation and down sampling the frames to target resolution.
It is good idea to render images at higher resolution than the target one is, down sampling in VirtualDub will smear jagged edges of 3D objects, in case you did not used antialiasing.
The filter dialog then might look like on the Pic 4.
The motion blur and interpolation can be done in ThinBASIC as well, but it would make no point when the VirtualDub does the job for us already. This way the script code can be kept relatively simple, and the routine work is done using third party tool.
Frame from final rendering can look similar to the Pic 5. You can see the lighting on UFO matches the lighting of the scene, and it is also visible the shake of background and the added object are in sync. The fog fading to atmosphere colour starts to manifest at this distance, helping to better integrate the UFO.
The produced movie can be downloaded for inspiration from here (15MB AVI) and the complete code is attached under the article.