User Tools

Site Tools


processing-golook-imagery

Processing GoLook Imagery

The Workflow

  1. Create a project folder with a sub-folder for GoLook Imagery.
  2. Copy the GPS Cube file to the Imagery folder.
  3. Copy each camera (1…7) to the Imagery folder as sub-folders 1…7.
  4. Start ScanLookVP and Process the Cube file.
  5. Copy each camera's video file into the new SessionX files.
  6. Process each or all sessions.
  7. Create output imagery (rectangular stills or pano video or stills).

Preparing the Imagery

Understanding the GoLook File/Folder Structure


Typically the ScanLook data structure starts with a Project folder. Under this we have an INS, Snoopy, Base, etc. folders for each type of data. While the names are not that important, consistency is and obviously a meaningful name is helpful.

With GoLook we need to add another folder. Use a meaningful name such as GoLook (Images, Imagery, etc.).

Within the GoLook folder we would put the Cube file. We would only put the single Cube file used for the particular imaging event. Once the Cube file is available, we can use Process Cube and have the rest of the folder structure created by the software. Here's what it looks like.

  • Project Folder
    • INS
    • Snoopy
    • FARO
    • Velodyne
    • GoLook
      • Folders 1 to 7 (direct from SD cards, temporary)
      • Cube File
      • Session1
        • Supporting Text Files (events, kml, etc.)
        • Folders 1 to 7
        • Inside each Session File we have:
          • One MOV file
          • Generated files
      • Session2 to Session 7

Copying the Image Data from the SD Cards to Windows


Once the Cube file is copied and Process Cube has been executed the GoLook data folders are ready to receive the image data (videos). This is a somewhat tedious process depending upon the number of projects and sessions on the cards. For this reason, we recommend starting each day with a clean set of SD cards. Of course, each camera SD card should contain the config.txt file. If your config.txt file should get deleted, either copy it from another card or from right here.

Here's a sample config.txt file for the camera SD cards:

MODE=VIDEO
PHOTO_INTERVAL=2
PHOTO_QTY=1
CONTINUOUS_MODE=OFF
VIDEO_CHUNK_LENGTH=600
SETTING=OUTDOOR
EXPOSURE=1.0
DISPLAY_FIRMWARE=ON
DISPLAY_HACKHD=OFF
DEFAULT_START_TIME= 07062015.0100
AUDIO_VOLUME=2
VIDEO_QTY=1000
PHOTO_RES=HIGH

Now you can copy your video data manually by following this example.

Copy SD1 HHD001.mov to Session1\1\HHD001.mov
Copy SD1 HHD002.mov to Session2\1\HHD002.mov
...
Copy SD7 HHD003.mov to Session3\7\HHD003.mov

Your GoLook folder now has folders 1 to 7, as shown above.

Moving the files in this manner can be tiresome, especially if you turn the cameras on and off frequently (which is a good idea). To help in this manner we would recommend using a simple batch script (moveImages.bat) to quickly move the MOV files into their proper folders. The script below MOVES data from 5 sessions worth of data into the appropriate folders (as Session1\1, etc.). The first session for each camera started at file 8 hence SessionOffset=7.

@echo off
REM Loop over each session (%%i)
REM Loop over all 7 cameras

REM Each folder has file *8 to 12.
REM There are 7 folders.
REM Move (1 to 7)\hhd*(8 to 12).mov to session(1 to 5)\

Setlocal EnableDelayedExpansion

set NumCameras=7
set NumSessions=5
set SessionOffset=7
for /L %%i in (1,1,%NumSessions%) DO (
	set /a fileIndex=%%i + %SessionOffset%
rem	echo %%i %SessionOffset% !fileIndex!
	for /L %%j in (1,1,%NumCameras%) DO (
		move %%j\hhd*!fileIndex!.mov session%%i\%%j
	)
)

pause

The ScanLook Video Processor (VP) Dialog


Generating the output imagery (individual rectangular frames, panoramic video or panoramic frames) is done using the ScanLookVP software. The single dialog that controls this process is shown below.

The main sections of this dialog consist of the Mission Processing, Session Processing, Thinning, and Panoramic Imagery. These will be discussed below.

Mission Processing


The Mission Processing section controls the processing of the main mission files, particularly the Cube file (cube###.txt). After copying the Cube file from the GPS card on GoLook and locating it on the Cube File field of the dialog, you need only select Process Cube to generate a session folder for each time the camera recording was turned on/off. Not only will these folders be created but each will be populated with files particular to that session as well. This includes a kml and events file per session.

The kml file (named cube###.kml) is available for immediate use in Google Earth and other programs.

The events.txt file contains gps time and position on an even second boundary per the recorded interval. Here's a sample file.

Event Time Week X Y Z O P K
1 444497 1864 666749.145078484 1521401.60974959 -7.9 0 0 0
2 444498 1864 666749.145078484 1521401.60974959 -7.9 0 0 0
3 444499 1864 666749.145078484 1521401.60974959 -7.9 0 0 0
Important note: The default coordinate system for GoLook is UTM meters.

Now let's take a look at each field in this section of the dialog.

The Cube File field, as already mentioned, is used to define the particular Cube file (GPS data file) for this mission.

Note: A new cube file will be created each time GoLook is turned on/off. 
  It is a good idea to start each day with a blank SD card.

The Process Cube command is used to generate the GoLook file and folder structure and create the necessary events and kml files for each video session.

The REM command simply removes the selected project from the list.

The UTC Zone defaults to the computers current time settings. If the data was collected in a different time zone then either change the computers time zone settings or select the proper values here (per hour and quarter hour).

The Reset to Local command simply returns the time zone settings to the computers settings.

The INS File may or may not be used. Generally it is the text file used in ScanLookPC for the system trajectory. It can be in the users coordinate system of choice.

The INS KML File is also optional. This is used to help generate a corrected kml file. Since the kml file from GoLook is generated using the low cost GPS onboard the camera, the KML file can be corrected significantly using the ScanLook generated kml file. At the moment it assumes this file has data on every even second (at least).

The Default SLVV file is used to denote the camera calibration parameters that are to be used for all subsequent processing steps. Typically this will point to a default calibration and should be replaced by your specific camera calibration.

If the INS File or KML file is updated at a later point, the Process Cube command can be run again to update the GoLook trajectory and KML file.

Session Processing


Each time the GoLook camera recording is turned on/off a new session will be created. On each camera SD card this is directly related to a new MOV file being generated. As we have already seen above, each MOV file is moved into its corresponding Session and camera folder.

The Video Folder list contains an entry for each session in the mission. Whatever is selected here is used for all remaining operations.

Once a video session is selected it can be further processed or viewed.

One of the first things to note is the seven (7) options for the Cameras to Use. This particular option allows you to select which combination of cameras are necessary for further processing. Normally it would be all 7 but it could be 5, or 3, or even 1. In any case, you can enable/disable individual cameras at any time.

The Inverted option is necessary to flag upside down mounting of the GoLook camera system. Sometimes it is necessary to mount it in this way.

The Leftmost option is used to designed which of the seven cameras will appear in ScanLookVV as the leftmost camera.

The Circular View option flags ScanLookVV to go beyond 360 degrees and actually use the leftmost camera as the rightmost camera as well.

Launch Player will launch the ScanLook Video Viewer. This is our own custom video viewer that is quite powerful. This can be launched at any time after loading the first video into any session folder. Simply enable the appropriate Cameras to Use and click on Launch Player. If you have already done a Process Session command then ScanLookVV will be able to geo-reference the video on a map as you play the video. ScanLookVV is also used to generate a refined camera calibration.

The three drop-down list should be set prior to launching ScanLookVV as they control the panoramic scene generation parameters.

The first paramoric parameter controls the Projection Model used to generate the panorama's. The default is Spherical. There are several interesting combinations that make for some good presentations to customers.

Here is a complete list of supported projection models:
Rectilinear
Fisheye
Stereographic
Mercator
Cylindrical
Spherical
Transverse Mercator

The second parameter has to do with the Seam Finder Algorithm. The default is gc_color.

Here is a complete list of supported seam finder algorithms:
voronoi
  Voronoi diagram-based seam estimator
gc_color
  Minimum graph cut-based seam estimator.
gc_color_gradient
dp_color
dp_color_gradient
none
  Apply no seam finder algorithm.

See http://docs.opencv.org/3.1.0/d9/d24/group__stitching__seam.html#gsc.tab=0 for a further discussion of each of these options.

The third parameter has to do with the Exposure Compensation Algorithm. The default is gain_block.

Here is a complete list of supported exposure compensation algorithms:
no
  No compensation will be applied
gain
  Exposure compensator which tries to remove exposure related artifacts by adjusting image intensities
gain_blocks
  Exposure compensator which tries to remove exposure related artifacts by adjusting image block intensities

See http://docs.opencv.org/3.1.0/dd/dff/group__stitching__exposure.html#gsc.tab=0 for a further discussion of each of these options.

For a thorough definition of the above parameters, see the OpenCV documentation found here: http://docs.opencv.org/3.1.0/index.html#gsc.tab=0.

The options chosen all affect the processing time. Some are very aggressive and are computationally expensive while others are not quite so time consuming.

Reset can be used to reinitialized the panoramic scene selections to the recommended settings.

The empty text box above the Reset command allows the user to alter the camera offsets by seconds for each camera. For instance, if camera 2 is 3.5 seconds behind the other cameras (for some crazy reason), then the field should simply contain:

0,3.5,0,0,0,0,0

The KML box will invoke the system KML viewer with the GoLook sessions KML file.

The SML box does the same as the KML box but it uses the KML file adjusted using the INS KML File from the Mission Processing. The S in SML is for ScanLook, simply replacing the K in KML.

The Process Session command button generates the event files for the selected cameras using the GPS Cube file and each video file. This can be a fairly time consuming project. All settings currently on the dialog will be used during this processing.

The All Sessions command button is similar to the Process Session button except it will quite literally process each session. This can be a big time saver if you have many sessions in a particular mission.

The Graphics button displays the current trajectory and allows for boundary selection (which is not yet implemented).

The Stop button interrupts whatever process may be running at the time.

Note: This part of the dialog is still under construction. There are a few things missing:
  1) Boresight
  2) Area Selection
  3) Re-organization

Thinning


Since GoLook collects real video at 30fps (frames per second), it is quite likely that the final product does not need every frame and should be thinned. This often results in a drastic file size reduction.

When we thin our input video into output frames we also have to have a file Format.

The most common formats are supported as:
jpg
bmp
png
gif
tif

If jpg is selected as the format, then the Q-factor fields are presented. The Best setting gives the least compression with the least loss of data whereas the Worst gives a maximum compression with some image loss.

The Filter Distance, the most important feature of the thinning process, is in project units (feet or meters). This parameter controls the ground interval at which image data will be extracted. All seven cameras are synchronized together to generate images at the same location.

The Run VG Thin command actually runs a batch (bat) file to create a Virtual Geomatics-formatted data file while also extracting the imagery from each video at the specified thinning interval.

The Run iPano, essentially obsolete, generates an image-butted (non-resampled) panorama using IRFanView. IRFanView must be installed for this to successfully run. It is a free image viewer and can be easily downloaded from the internet.

The Run Thin command will run the lengthy image extraction at the thinning interval. Within each camera folder there will be a resulting Frames folder containing the output. Each folder will also have a corresponding image index file.

The Thin command is only necessary if the Format or the Filter Distance has changed otherwise during the Process Session the thinning is already computed. The actual thinning process is very time consuming but this command only computes the thinning parameters and is not very time consuming.

The By Location field is not yet fully implemented. The intention is to generate image data at certain locations (XYZ) as specified by the file entered into this field.

Panoramic Imagery


This section controls the panoramic video and still image generation.

The Videos2Pano installation must have been completed successfully as part of the ScanLookVP setup or these commands will not work.

Creating a Pano Video

The Create Video command invokes a very lengthy process once activated. It's purpose is to generate a panoramic video. Here's how it works.

The SLVV specified in this section (directly below this button) is used to provide many of the necessary parameters for the video creation (events, panorama refinements, etc.). When executed, a new folder in the selected session will be created with the name as <SLVV base file name>_<SCALE>. So if the SLVV file name is c:\project\session1\camera.slvv (the default created for the session) with an image scale of 100, then the folder name will be camera_100.

If All Frames is checked then the entire input video is used to create the output video (no thinning). If it is not checked then only the frames specified in the event files as pointed to be the slvv file will be used.

To effectively create a thinned video, All Frames must not be checked, and the thinned.slvv must be selected. This can result with a thinned video with no stops thereby drastically reducing the video file size.

The Frame Rate field is used to set the default frame rate in the resulting video file. Depending upon your distance interval and purpose this can be anywhere from 1 to 30 fps.

Checking the Mov option results in an MOV video file being created after clicking on Create Video.

Checking the MKV option results in an MKV video file being created after clicking on Create Video. This can then be easily converted and used in 3D viewers. A simple search for “youtube 360 video” will provide ample methods of uploading your pano's to Youtube for 360 viewing (and display with Google Cardbord, etc.).

Extracting A Video Segment

The Range box allows the entry of begin:end frames (i.e. begin and end a frame numbers from ScanLookVV separated by a colon). Using ScanLookVV the user can determine the begin:end video frames they may want to extract to a separate panoramic video. These values would be entered into this box.

The image below shows a starting position of 1000.

If you want to extract a short video clip as a pano then check All Frames, MOV, and enter a Range. In the SLVV field be sure to specify a good slvv file (one that generates nice panos). Select the Scale and Frame Rate.

Then click on Create Video.

The output will be in the same folder as the input in a folder named as <path>\<SLVV Base filename>_<Scale>. The output folder will have event files (events_finalized.txt and a GeoFinalized.txt), the output MOV (HHD0001.mov or .mkv), and the slvv file (HHD0001.slvv). These can be packed up and shared with anybody. Just give them a setup for ScanLookVV and away they go.

Creating Pano Still Images

The Create Stills command will generate panoramic images (in individual files, not in a video) as a function of the Scale, Crop, Image Format, and thinning parameters. The imagery will be in the Session folder called <Frames>_<SCALE>.

The thinned.slvv file for the session will automatically be used at that last computed thinning interval.

The events_pano.txt, GeoFinalized.txt and pano_index.txt files will be copied to the output folder defined above. These are geo-referencing and index files for the output imagery.

Finally, the output image names will be pano_<Event>.jpg.

Here's a sample of what an events_pano.txt file looks like:

000001 444497.004980 1864 666749.145 1521401.610 -7.900 .000000 .000000 .000000
000002 444628.303709 1864 666749.936 1521421.925 -5.679 .000000 .000000 .000000
000003 444631.837021 1864 666751.507 1521441.974 -3.349 .000000 .000000 .000000

Here's an example of what a GeoFinalized.txt file looks like:

218 444497.004979667 1864 100.542333 13.757143 -7.9 0 0 0
4157 444628.303708667 1864 100.542341518543 13.757326578016 -5.67851653333334 0 0 0
4263 444631.837021 1864 100.542357229357 13.757507710197 -3.3488937 0 0 0

Here's a sample of what a pano_index.txt file looks like:

000001 444497.004980 1864 666749.145 1521401.610 -7.900 .000000 .000000 .000000 pano\000001.jpg
000002 444628.303709 1864 666749.936 1521421.925 -5.679 .000000 .000000 .000000 pano\000002.jpg
000003 444631.837021 1864 666751.507 1521441.974 -3.349 .000000 .000000 .000000 pano\000003.jpg

Both the commands create supporting index files with geo-referencing information easily assimilated with the resulting output.

The Scale selection can be 25, 50, or 100. These are percentages at which the output should be scaled. Obviously 100 will be a full resolution image and will generally be the most time and space consuming option.

The Crop option shortens the image height of the panoramic by removing all black filled areas. It is not generally recommended to use this option.

processing-golook-imagery.txt · Last modified: 2016/03/24 15:29 by jeff.fagerman

Bitnami