GIS Tools

ASF ArcGIS Desktop SAR Toolbox

ASF has developed an ArcGIS Desktop Toolbox containing tools that perform geoprocessing tasks useful for working with Synthetic Aperture Radar (SAR) data. The tools were designed to be used with Radiometric Terrain Corrected (RTC) SAR datasets, but several of the tools have the potential to be used with a variety of rasters, including non-SAR datasets. The Toolbox is distributed as a zipped archive including the .pyt Toolbox script and associated .xml files. There is an XML file for the toolbox itself and one for each of the tools it contains. These XML files are accessed when a user views the Item Description for the toolbox or any of its constituent tools in the ArcMap Catalog window. The information can also be found in the Description tab in ArcCatalog, and portions of the xml contents are displayed as appropriate in the tool dialog box when a tool is launched in ArcMap. These XML files must be kept in the same directory as the Python Toolbox (.pyt) file, or the information they contain will no longer be accessible to ArcGIS.

The Toolbox currently contains the following Tools:

  • Unzip Files Tool: assists in file management when downloading .zip files from ASF. This tool could be used to extract to a specified location any zip files with an additional internal directory containing the individual files. The tool deletes the original zip files once they are extracted, and is especially helpful when dealing with filenames that are so long that they are beyond the maximum allowed in default Windows unzip utilities.
  • Scale Conversion Tool: allows easy conversion of pixel values in calibrated SAR datasets (such as RTC rasters) from power or amplitude scale into power, amplitude or dB scale. This is an application specific to SAR data values.
  • Reclassify RTC Tool: generates a raster that includes only those pixels below a user-defined threshold value, and is designed for identifying water pixels. This tool could be used for any application where the user is interested in identifying values below a given threshold.
  • Log Difference Tool: compares two rasters by calculating the log difference on a pixel-by-pixel bases to identify areas where backscatter values have changed over time. This tool could be used to compare the pixel values of any two single-band rasters; it is not specific to SAR data.

Prerequisites:

  • Users must have ArcGIS Desktop (ArcMap) installed and licensed on their computer. The Toolbox has been tested with versions 10.6.1 and 10.7.1, but may work with earlier versions.
  • Note that several of the tools require the Spatial Analyst extension. Users who do not have licensing for this extension in ArcGIS will not be able to use many of the included tools.

To install the Toolbox:

  1. Download the zip file, and extract the contents to any directory accessible by the computer running ArcGIS.
  2. Ensure you have access to the Spatial Analyst Extension in ArcMap.
    • Click on the Customize menu in ArcMap and select Extensions…. Check the box next to Spatial Analyst and click the Close button at the bottom of the Extensions window. If you are unable to check this box, you do not have access to the Spatial Analyst extension and will not be able to make use of tools requiring this extension.

To use the Toolbox:

  1. In the ArcMap Catalog window, navigate to the directory containing the toolbox (create a new folder connection if necessary).
    • To open the Catalog window, click on the Windows menu in ArcMap and select Catalog.
    • Note that if you explore the extracted contents outside of the ArcGIS environment, the directory will contain one .pyt file and a number of .xml files. In the ArcMap Catalog window, only the Toolbox is displayed, and when it is expanded, each of the Tools contained in the Toolbox script is displayed. The XML files are automatically referenced when ArcGIS requires the information they contain, and do not appear as additional files in the ArcMap Catalog environment. The XML files must remain in the same directory as the .pyt file, and their filenames should not be changed.
  2. Double-click the ASF_Tools.pyt file to display the Tools (Scripts) included in the toolbox.
  3. Double-click on a Tool (displayed with a Script icon) to launch the dialog box, as you would for any other ArcGIS Tool/Script.
  4. Click on the Show Help button at the bottom of the tool window to see more information about the tool and helpful tips specific to each parameter.
  5. Click on the Tool Help button at the bottom of the Help pane to open another window that displays most of the contents of the tool’s Item Description.
  6. Enter the parameters as prompted and click the OK button to execute the tool.
  7. Note that output products are not automatically added to a project by default. You must navigate to them in the Catalog window (or using the Add Data dialog) and add them to your project if desired.
Contact ASF if you have questions about the Toolbox, or if you have suggestions for other tools to include.

Vertex Getting Started User Guide

  • If you do not already have one, create a free Earthdata Login account at https://urs.earthdata.nasa.gov/users/new.
  • Log in by clicking the Sign in icon in the top right of the window. Use your Earthdata Login username and password.
  • Search Type allows you to choose between Geographic Search and List Search.

Geographic Search Options

  • The Vertex map defaults to Satellite View
  • Navigate to your area of interest by dragging the map while holding down the left mouse button.
  • By default, the map-drawing tool is a bounding box. Click on the map once to specify the starting corner, move the mouse, then click again to finish the box. Additional drawing tool options are available in the toolbar at the top of the screen, including point, linestring, and polygon options.
    • Point allows you to define an area of interest by clicking on the map to place a point.
    • Line allows you to define an area of interest over a series of line segments by clicking on the map multiple times. Double-click to stop adding segments.
    • Polygon allows you to define an area of interest over an arbitrary polygon. You will receive an error message at the bottom of the window if there was a problem with the polygon (self-intersecting, reversed polygon winding order, etc.).
    • Box allows you to define an area of interest over a lat/long-aligned bounding box by clicking once to set one corner, and again to set the opposite corner.
    • Once a shape has been drawn, select the Edit current area of interest icon on the toolbar to move, add, and delete points. Select the Draw new area of interest icon to create a new AOI.
  • Dataset enables you to choose the dataset of interest.
    • If you need more information about a particular dataset, click on the appropriate icon in the Dataset selector.
  • Area of Interest gives you the option of importing an area of interest as a geospatial file or by entering a set of geographic coordinates.
    • Click on the down arrow next to Area of Interest in the top menu
    • Click the Import Area of Interest button in the Options window
    • Click Select Files and navigate to a folder on your computer, or drag and drop files into the box. GeoJSON, shapefiles, and kml files are supported provided they are in a latitude/longitude-based coordinate system, such as WGS84.
      • When importing a GeoJSON file, all geometries in the file will be included. If multiple geometries are found, a convex hull will be used to represent them in the search.
      • Shapefiles can be either a single .shp file, multiple shapefile components (.shp, .shx, .dbf), or a zip file containing one or more shapefile components. At a minimum, the .shp component must be included in all cases.
    • An area of interest may also be defined by a set of coordinates entered in the Options window.
      • Coordinates should be entered as decimal degrees in well-known text (WKT) format. Coordinates entered as a comma-separated long/lat string (e.g. -97.38,36.46,-53.44,36.46…) will be automatically converted by Vertex to WKT format.
  • At any time you can clear your search area by clicking the Clear button in the header.
  • Search dates are optional, so they default to empty.  If you are searching for specific dates, you can define the date range further in the Start Date and End Date fields. The date picker will automatically constrain your selection to a valid range for the selected dataset. Note: this information may also be found by clicking on for a dataset.
  • Filters… enables you to further refine your search
    • Path and Frame Filters are available for select datasets. You may enter a single path or frame, or a range. Due to inconsistent Sentinel-1 framing, we recommend searching for a frame of interest by ±1-2 frames.
    • Seasonal Search allows constraining the search to certain annual periods within an overall range of dates.  Click the Seasonal Search toggle and additional options will appear, allowing you to enter an overall date range (Start Date/End Date) and the seasonal range (Season Start Day/Season End Day).
    • Additional Filters allow for additional parameters to be applied to narrow your search and reduce the number of results. Not all filters will be available for all datasets.
      • File Type – Limit the search to specific types of files. Multiple selections allowed.
      • Beam Mode – Limit the search to specific beam modes. Multiple selections allowed.
      • Polarization – Limit the search to specific polarizations. Multiple selections allowed.
      • Direction – Limit the search to a specific orbit direction.
      • Campaign Selector – If a dataset is selected that supports this option, a campaign selector will be available. Only one selection is allowed. Currently, S1 InSAR (BETA), UAVSAR, and AIRSAR support this option.
  • Once all parameters have been chosen, click SEARCH. Search results will appear in the footer area of the Vertex window, and on the map.
    • Note: The number of files that are predicted to match the current search parameters is displayed under the SEARCH button. If there are no predicted matches, the search button will be greyed out and display NO RESULTS.

List Search Options

  • Selecting List Search opens the Scene List Search window and allows you to enter a list of scenes or file names.
    • Scene allows searching for specific scene names (granule names), and the results will include any files that are part of those scenes.
    • File allows searching for specific file names (product names), and the results will only include exactly those files.
  • Edit List – Opens the Scene List Search window so you can make changes to your list
  • Once all parameters have been chosen, click SEARCH. Search results will appear in the footer area of your browser window, and on the map.
    • Note: The number of files that are predicted to match the current search parameters is displayed under the SEARCH button. If there are no predicted matches, the search button will be greyed out and display NO RESULTS.

Search Results

  • Note: In Vertex, a scene is considered to be a package containing all files, or products, that are related to a specific location and time.

For example, the column on the left of the Results panel displays the scenes returned from a search. The column on the right displays the file contents of a scene.

  • A list of scene names and acquisition times that were returned from a search are displayed in the leftmost column of the Results panel.
  • The maximum number of files that a search will return is displayed under the SEARCH button.
    • This number can be adjusted by clicking on the down arrow.  
    • The total number of files that match the search parameters is also displayed.
  • The Scenes column (left).
    • Click on the cart icon next to a scene name to add all the scene’s files to the download queue. The cart changes appearance when this is done.
    • Click on the zoom icon next to a scene name to zoom-in to the scene’s location on the map.
  • To view more information about a scene, click on the scene in the left column and the Scene Detail and Files columns will populate.
    • The Scene Detail column (center) provides a more detailed description of the scene, including Start Date/Time, Beam Mode, Path, Frame, Flight Direction, Polarization, Absolute Orbit, and a browse image. Not all scenes will have all the extra information.
      • The Baseline Tool button opens the ASF Baseline Tool, which is used for the creation of InSAR stacks.
      • The Citation button opens a new window with citation guidance for published works using data, imagery, or tools accessed through ASF.
      • The More Like This button creates a search based on the selected scene’s path and frame.
    • The Files column (right) displays a list of files available for the currently selected scene. You may download files immediately or add them to your download queue by clicking on the appropriate icon.

Download Queue

  • Clicking on the cart icon in the header, labeled Downloads, will display the contents of your current download queue.
    • Within the download queue, the list of files you have selected to download is displayed with some basic information on each file, such as file type and size.
      • File IDs (names) can be copied
      • Files can be individually downloaded
      • Items can be removed from the queue
    • Clear Queue will clear all files from the queue. The option Restore Queue will be displayed if you wish to undo this action.
    • Copy File IDs will copy the file names of all files in the queue for use elsewhere. For example, this list could then be pasted into the Scene List Search window.
    • Data Download – Is used to download multiple products, with either the Download Python Script (.py) option or Metalink (metalink) file option.
    • Metadata Download – Is used to export the contents of the download queue to a CSV, KML, or GeoJSON file. The KML and GeoJSON files provided by this feature are compatible with the Geographic Search Import feature.

Saving Vertex Searches

  • You can save the coordinates of a search so they can be used to exactly recreate an area of interest in later searches.
    • Once the Area of Interest has been set, mouse over the coordinates. A Copy to clipboard icon will appear. Click on the icon and paste the coordinates into a new search or to a text file for later use.
  • All the parameters used in a search can be shared with others or copied and used again.
    •   Click on the horizontal bars menu next to the Sign in icon and select an option.
  • The search parameters will be saved in a URL that can be pasted into a browser search bar to recreate the search exactly.
Data Recipes — Further Reading

Data Recipes (Tutorials)

Recipes for Using SAR

  • All
  • ASF MapReady
  • ArcGIS
  • Cloud Computing
  • GAMMA
  • GDAL
  • GMT5SAR
  • InSAR
  • QGIS
  • Radiometric Terrain Correction
  • SNAPHU
  • Sentinel-1 Toolbox
  • All
  • ASF MapReady
  • ArcGIS
  • Cloud Computing
  • GAMMA
  • GDAL
  • GMT5SAR
  • InSAR
  • QGIS
  • Radiometric Terrain Correction
  • SNAPHU
  • Sentinel-1 Toolbox
Script output images: color phase products for each swath F1, F2, F3. Contains modified Copernicus Sentinel data (2015) processed by ESA.

How to Configure AWS for Running the GMT5SAR InSAR Recipe

Script output images: color phase products for each swath F1, F2, F3. Contains modified Copernicus Sentinel data (2015) processed by ESA.

Adapted from instructions prepared by C. Stoner, ASF

Easy (AWS sign-up and configuration)

Background

After logging-in the user must make some choices before creating their InSAR products. These steps guide you through that process, with recommended settings.

Steps

Start an EC2 Instance

    1. Sign in to your AWS account, or create one now.
AWS Management Console
    1. In AWS Services, under Compute, click on EC2.
Navigate to EC2

Select an Amazon Machine Image (AMI) — i.e., the software package you want

    1. At the right on the top menu bar, set the Region to US East (N. Virginia) by selecting from the drop-down menu. 
    2. Under Create Instance, click on the Launch Instance button.
Select the Region and Launch Instance
    1. Choose an Amazon Machine Instance (AMI) — Step 1
      • (1) Under Quick Start, click on Community AMIs.
      • (2) In the Search community AMIs box, type “asf-insar-gmt5sar” and press Enter.
      • (3) Click the Select button next to ASF-INSAR-GMT5SAR
Search Community AMIs for asf-insar-gmt5sar
Select the ASF-INSAR-GMT5SAR AMI

Configure the EC2 Resources

    1. Choose an Instance Type — Step 2
      • Scroll down the table of available instances and choose: m4.xlarge.
        • Then click on the Next: Configure Instance Details button.
    2. Configure Instance Details — Step 3
      • Nothing needs to be changed here, so click on the Next: Add Storage button.
Choose Instance Type m4.xlarge
View Instance Details
    1. Add Storage — Step 4
      • The volume storage size is preset, so confirm that the Size (GiB) value is 230 and then click on the Next: Add Tags button.
Set Size value, then click Next: Add Tags
    1. Add Tags — Step 5
      • (1) Click on the Add Tag button.
      • (2) Under Key, type “Name.”
      • (3) Under Value, type a description; for example, “asf-insar-tutorial.”
      • Click on the Next: Configure Security Group button.
Click Add Tag
Enter text for the Key and Value entries
    1. Configure Security Group — Step 6
      • This step allows you to set access permissions for your instance. We will create a setting so that only your computer can access your instance.
      • In the Security group name box, type a name.
      • Under Source, click on Custom and select and click on My IP.
      • The IP address of your computer is automatically selected.
      • Click on the Review and Launch button.
Enter a Security group name and choose a Source
Select My IP as a Source
    1. Review Instance Launch — Step 7
      • This last step allows you to review the details of your configuration before activating your new instance.
        • A warning is displayed at the top of the screen that can be ignored. The “free usage tier” AMIs offered in AWS are not powerful enough to process the GMT5SAR InSAR recipe.
      • Click on the Launch button.
Review the details of your configuration

Create a Key Pair (.pem file)

    1. Create a key pair for authentication and encryption when connecting to your running EC2 Instance. Read more on information about key pairs.
      • Click on Choose an existing key pair and select Create a new key pair.
      • Name the key pair; for example, asf_tutorial_keypair.
      • Click on the Download Key Pair button.

Note: The key pair file will download to your default Download directory as filename.pem and can be moved to a directory of your choice. The .pem file is required to connect to your EC2 Instance and to transfer data between your computer and your EC2 Instance.

Select Create a new key pair
Download your Key Pair

Launch the Instance

    1. After creating the key pair, click on the Launch Instances button.
    2. The Launch Status window provides information on your new EC2 Instance. To view your instance, scroll to the bottom of the page and click on the View Instances button.
View Launch Status
View your EC2 Instance's details

Terminate the Instance

Important: Do not terminate your EC2 Instance until you have finished processing and moved any files you want to save from your instance to your computer. To connect to your EC2 Instance and move files in or out, refer to the ASF Data Recipes: How to Connect to an AWS EC2 Instance — OS X and How to Connect to an AWS EC2 Instance using PuTTY v1.1 (Windows)

    1. Once you have launched your instance, you will start incurring charges on your account.
    2. When you have finished processing, you need to terminate the EC2 instance to avoid incurring additional charges.
      • From the EC2 Dashboard, navigate to Actions > Instance State.
        • Click on Terminate and confirm.
        • This deletes the instance and all data stored on the instance; a new AMI will need to be configured for any future processing.

Important: Selecting Stop will shut the instance down and stop further EC2 charges from accruing. But you will continue to be charged for the data left in the instance EBS storage volume (e.g., the GMT5SAR script, granule files, PRODUCT files, etc.) The advantage of this option is that you can restart the instance and avoid configuring a new one for additional processing. 

Note: Each time an instance is started, a minimum one minute is charged. After one minute, you are charged by the seconds used.

Navigate to terminate the EC2 Instance