Skip to content
Uhurulabs
  • ICT Consultancy
    • Data backup and security
    • Disaster Recovery
    • Infrastructure design and implementation
    • Data Processing
  • UAV Services
    • Survey & Mapping
    • Agriculture
    • Mining, Quarries & Aggregates
    • Engineering & Construction
    • Training
    • Permit Application
    • Our Tools
  • Projects
    • Lake Victoria Challenge
    • Building Drones
    • 3D Printing
  • About us
    • Mission and Vision
    • Meet the Team
      • UAV Pilots
    • Contact us
  • Blog

Trying to merge lots of big GeoTIFF’s

  • 4th April 20172nd January 2019
  • by Frederick Mbuya

The idea was to merge into 1 geotiff 28 irregular shaped geotiffs which ranged in size from 1g up to 13g and making up a total of 113g. Why? Because I then needed to cut the resulting geotiff into multiple irregular shaped individual geotiff’s. This took a couple of days, and I was quite keen to come home today as this morning it was at 80%, you can only imagine my disappointment when I checked and found…

0...10...20...30...40...50...60...70...80...90..Traceback (most recent call last):
  File "/usr/bin/gdal_merge.py", line 540, in <module>
    sys.exit(main())
  File "/usr/bin/gdal_merge.py", line 526, in main
    fi.copy_into( t_fh, band, band, nodata )
  File "/usr/bin/gdal_merge.py", line 270, in copy_into
    nodata_arg )
File "/usr/bin/gdal_merge.py", line 63, in raster_copy
    nodata )
  File "/usr/bin/gdal_merge.py", line 105, in raster_copy_with_nodata
    nodata_test = Numeric.equal(data_src,nodata)
MemoryError

This as you can imagine was very frustrating, and I just assumed that the output was junk but figured what the hell, its created a 301G file lets see what it is. Started the process to load into Qgis, and since I figured it would take a while started this blog post. It seems though that output might be useful as qgis eventually loaded the file, and it actually looks like what I expected. So first things first I have set qgis to now save the file under a new name… Its going to take a while, currently at 12.2G and I actually expect it to end up bigger than the initial 301G

… a day or so later …

So it turns out the merge worked, I will revisit why the error later, but the file saved by qgis had the exact same size, and has the same result using gdalinfo. AND i have just done my first clip BUT  it seems I made a mistake the command I used was

gdalwarp -dstnodata 0 \
-q -cutline shape_file_to_clip_to.shp -tr 0.05806 0.05806 \
-of GTiff input_file.tif \
clipped_file.tif

Which resulted in a file the same size of the input file, and the same dimentions, what I should have done, I think, is:

gdalwarp -dstnodata 0 \
-q -cutline shape_file_to_clip_to.shp -tr 0.05806 0.05806 \
-of GTiff input_file.tif \
clipped_file.tif

Which is what I am running now, the previous command took about 3 hours!

Share this:

  • Click to share on X (Opens in new window) X
  • Click to share on Telegram (Opens in new window) Telegram
  • Click to share on Pocket (Opens in new window) Pocket
  • Click to share on Pinterest (Opens in new window) Pinterest
  • Click to share on Tumblr (Opens in new window) Tumblr
  • Click to share on Reddit (Opens in new window) Reddit
  • Click to share on Facebook (Opens in new window) Facebook
  • Click to share on LinkedIn (Opens in new window) LinkedIn
  • Click to share on WhatsApp (Opens in new window) WhatsApp
  • Click to print (Opens in new window) Print

Related

GeoTIFF Transparency
New Firewall
egm

Related articles

Some OpenData
The Zanzibar Mapping Initiative
Why I am so disapointed…
Working on the edge…
Sustainia 4th Global Opportunity Report
Drone Journalism School
Citrix XEN to Linux KVM
How safe do you feel?
New Firewall
GeoTIFF Transparency

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recent Posts

  • Some OpenData
  • Gymkhana
  • The Zanzibar Mapping Initiative
  • Why I am so disapointed in the Black Panther Movie
  • Working on the edge…

Recent Comments

  • Frederick Mbuya on GeoTIFF Transparency
  • Frederick Mbuya on GeoTIFF Transparency

Archives

  • January 2019
  • October 2018
  • March 2018
  • February 2018
  • September 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017

Categories

  • Blog
  • GeoTIFF
  • GIS
  • UAV Related
  • Uncategorised
  • Virtualization

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

Follow me on Twitter

My Tweets
Theme by Colorlib Powered by WordPress