HOW TO CREATE AND VISUALIZE CLOUD OPTIMISED GEOTIFFs (COG)
A Cloud Optimized GeoTIFF file, otherwise known as COG, is a regular GeoTIFF that allows a user to request a specific part of the file relevant to the analysis. In simpler terms, you can stream a GeoTIFF into your GIS software without downloading it to your machine.
Let’s talk a bit about the tech behind COGs and the different platforms where you can make such requests. Then I’ll walk you through the process of importing Cloud Optimized GeoTIFFs into the QGIS long-term release and finally, how to create a COG using the GDAL command line.
- COGs work by storing images in tiles and overviews and retrieving them using HTTP GET Range requests.
- Data providers help users search POI and retrieve download links.
- To visualize COG in QGIS 3.2 and later, navigate to Data Source Manager >> Raster >> Protocol HTTP(S), cloud, etc. >> Type: HTTP/HTTPS/FTP >> Paste link >> Add > Close.
- To visualize COG in earlier QGIS versions, navigate to: Raster >> Miscellaneous >> Build Virtual Raster (Catalog). Prepend the copied URL with /vsicurl/ before pasting the link in the input files field
- To convert your regular GeoTIFF to a COG, install the OSGeo4W Shell and run the ‘gdalinfo’ and ‘gdal_translate’ commands to view and convert respectively.
How Cloud-Optimized GeoTIFFs Work
COGs work on three technologies;
1. The GeoTIFF stores the raw pixels of the actual image as an index of ‘tiles’, making it easier to read specific portions of the file.
2. It organizes the pixels into multiple ‘overviews’, representing different zoom levels. This makes selecting a point of interest quicker as only a lower quality is rendered.
3. It implements the HTTP Get range requests allowing a user to request portions of the image file.
The first two make the latter possible.
Sources Of Cloud Optimized GeoTIFFs
You can obtain Cloud Optimized GeoTIFFs from the following data providers: OpenAerialMap, SpaceNet, Mundi, Planet, ISERV, CBERS, and DigitalGlobe’s GBDX.
In this tutorial, we will work with a COG from OpenAerialMap. Browse through its database to find a suitable image for your point/data of interest and copy its download link. The link is temporarily saved to your clipboard.
That is: 1Search >> 2Download (Right Click) > 3Copy Link Address
1 copy download link from OpenAerialMap
You can choose to save the copied link to a notepad for easy access.
Another source worth trying is the Mundi platform. It provides some of its data from Sentinel-1 satellites as COGs. It offers search flexibility as users can adjust time scales and apply preferred filters and layers to downloaded maps.
How To Visualize COGs to QGIS Version 3.2 and later
The Data Source Manager tool allows you to import different data types into your QGIS workspace – Can’t find it on your toolbar? Right-click anywhere on the 1toolbar and tick the 2Data Source Manager checkbox.
2 Activate Data source manager tool on QGIS
On the open dialogue box, set the data type to raster and source type to Protocol HTTP(S), cloud, etc. Set the Protocol Type to HTTP/HTTPS/FTP and paste the copied link from OpenAerialMap in the protocol link field. ‘Add’ the file when done with the setup and ‘Close’ the dialogue box.
I’ll stick with the default settings for the Authentication configuration.
1Data Source Manager >> 2Raster >> 3Protocol HTTP(S), cloud, etc. >> Type: 4HTTP/HTTPS/FTP >> 5Paste link >> 6Add > 7Close
3 import and visualize Cloud Optimised GeoTIFFs on QGIS 3.2+
How To Visualize COGs to QGIS Version Pre 3.2
Reading a cloud Optimized GeoTIFF to earlier versions of QGIS isn’t as straightforward. As you might have noticed, an attempt to import a file automatically opens your storage directories. The workaround? GDAL. Install the GDAL tools Plugin (This is a core plugin and is typically installed)
Plugins >> manage and install plugins >> search >> GDAL > Install
With the aid of the GDAL tools plugin, we will create a Virtual Raster file (VRT), this will help read our COG using the copied link.
To do this: 1Click on Raster >> 2Miscellaneous >> 3Build Virtual Raster (Catalog).
For a more in-depth review of Virtual Rasters click here
4 Import and visualize Cloud Optimized GeoTIFFs in older versions of QGIS
On the dialogue box, Prepend the copied URL with /vsicurl/ before pasting the link in the 1input files field as shown below. Specify your 2output file where you would want your COG pointer saved.
Here, I chose to work with the defaults. Feel free to play around with the ‘Resolution’ and ‘Source No Data’ options. The latter assures you of a transparent background. Check the ‘Load into canvas when finished’ field and your imported raster file will be displayed on the workspace after processing. When done, click > 3OK
5 Setting up the Build Raster (Catalog) on QGIS
You can point to multiple COGs in a single VRT by;
- Prepending each link with /vsicurl/
- Separating each link with a comma (no spaces)
How To Create A COG Using GDAL Command Line
One of the easiest ways to create a COG is using the GDAL command line. Here, I put together a 6-step process for converting your regular GeoTIFF to a COG using GDAL. I’ll be working with a Lansat-8 image with the file name: LC08Band1.
Step 1: Install OSGeo4W Shell
Download and install the OS4Geo network installer from the QGIS website. Check here for a detailed guide. Run the shell and proceed to step 2.
Step 2: Confirm GDAL Installation and Version
Confirm your installation using the gdalinfo command. This displays your GDAL version and released date as in the image. Type: gdalinfo –version
Step 3: View the GeoTIFFs Metadata
Checking out the metadata gives you a better idea of what you’d be working with. You can do this by going into the directory of your data and using gdalinfo filename.TIF
From my metadata, I see the compression type is LZW (more on that). You can also take note of the INTERLEAVE and projection frames if you intend making any adjustments.
Step 4: Convert to COG
Here, I’ll use the gdal_translate command to optimize the file. Type: gdal_translate filename.TIF newfileCOG.TIF -of COG -co COMPRESS=LZW
Be sure to make sure your creation options match your image metadata. In my case, I set compression to LWZ. See the GDAL COG documentation for other creative options.
Step 5: Confirm the Creation
Let’s see if this worked. We can do this using other sources like cogeo.org or GDAL Wiki. Instead of using any validation scripts, we can easily check the image structure metadata section of the new file. If the layout structure is a COG, then it is a COG complaint. gdalinfo newfileCOG
The new file is in the same directory as the old one.
Step 6: Upload to AWS Servers
Of the available web services, Amazon provides an easy setup process. Check out their tutorials page for how to sign up for an AWS account or its free version. See the guide on getting started for directions on how to create an S3 bucket, create a folder, and upload your new COG.
Final Thoughts On COG’s
Working with Cloud Optimize GeoTIFFs is as fast as it is convenient. I hope this answers your questions. Use the added backlinks for additional information if necessary.