Higher Quality UTM Zones Map

Until now, an image from the Wikipedia article on the UTM coordinate system has been my go-to UTM Zone reference. It's pretty okay.

I threw together a map that has been a bit more helpful as it shows US counties and some basic hydrography -  mostly from ArcGIS Online layers, and default ArcMap Template data.

Download the PDF here

Add a Web Mapping Data Service to ArcMap

I have a hole in my current coverage of waterbodies - I need to know the name of a few streams, and the dataset of Rivers doesn't include the stream I need. I don't necessarily want to download and manage another whole dataset of flowing waters. Temporarily referencing a Web mapping service in my ArcMap project would be great.

The best database I am aware of that includes comprehensive waterbody geometry and names is the USGS's National Hydrography Dataset. Thankfully, they do indeed provide a Web mapping service!

Search for a Mapping Service

A quick Google search with the phrase nhd web map returns a direct link to the USGS National Map's map service endpoint:

http://services.nationalmap.gov/arcgis/rest/services/nhd/MapServer

• Copy this URL from the browser for use in the next section

These Map or Feature Service URLs (referred to as "service endpoints" or simply "endpoints") aren't always so easy to find, so other projects/data services may require more digging. Look for clues in the URL itself such as the /rest/services/ directories, and for minimal styling (basic-looking fonts, colors, lists, etc.).

To link to a specific portion of the map service (for instance a specific layer or layer group within the main service layer), follow some of the links under the Layers list (see below). Copy that specific URL for use in the next step if you would like. Compare the following URL that represents the Waterbody layer which is layer position 11 in the service package:

http://services.nationalmap.gov/arcgis/rest/services/nhd/MapServer/11

Add a Service to ArcMap

1. Click the Add Data button 
2. Change the Look in category to GIS Servers

3. Double click on Add ArcGIS Server to launch the Add ArcGIS Server wizard
4. Accept the default value to simply Use GIS services and click Next
5. Paste the Server URL that was copied from where you found the chosen service endpont
6. Leave the User Name and Password blank if the server is open for public use, and click Finish

Adding the Service Layer to an ArcMap Project

• Hit the Add Data button again if you aren't already there
• Ensure your Look In category is pointing to GIS Servers, just as we did in the last section
• Double click on NHD on services.nationalmap

! Note: Slow double click on this element to rename this to "National Hydrography Dataset" or anything else if you don't prefer this title

• Double click the nhd service layer to add the dataset to your project

! Note: This is a large dataset with that's referencing a ton of layers and information. Depending on the speed of your computer, your network connection, and the size of your existing ArcMap project, it may take some time for the layer to be added to your project.

Structure of the NHD map service. Notice that scale-
dependent drawing is set for a number of layer groups.

Make New Project Directory

I like to organize photos, videos, GIS projects, and developments projects in separate "project directories." Each project folder has the following format:

YYYY-MM-DD - Description

File Structure Example:

• [Drive]:\
• Development
• 2014-10-04 - Scuba Calculators
• 2015-06-01 - Make New Project Directory
• GIS
• 2012-05-14 - Whatever Project...
• Shapefiles
• Exports
• From Client
• MXDs
• Photos
• 2013-04-05 - Birthday in Midtown
• Hilights
• Edited
• Original
• ...
• Videos
• 2015-06-01 - Michelin Man Thug Life
• Source Media
• Exports
• ...

This is an easy way to chronologically catalog projects within their respective folders, but I'm lazy, and I hate having to look down at my task bar clock to find out what the date is, and I can't stand typing in that endless date!

Solutions Posted on GitHub

I posted all the scripts I came up with for this on my GitHub page under the Make New Project Directory repo.

Batch File

I originally tried coming up with a Batch file (.bat) to add the folder using the format I wanted. It's easy to make a predefined folder structure using CMD commands, but pulling that date style turned out to be a little tricky, so I looked elsewhere.

Python Script

I came up with a great Python script that will automatically add a folder that's named with the current date followed by XXXXDescriptionXXXX (so I could easily find and rename it further). Just double click the python file and the new project folder is added. Additionally, I repeated the code to add subdirectories for common sub-folders: Exports and Shapefiles for GIS projects; Originals, Highlights, and Edited for photos & video; etc.

Pros:

• This is great! It works automatically
• Simple code. It executes in 6 lines.  View the code here

Cons:

• I quickly realized how awfully tedious it is to have to slow double click to rename, something every time I create a new project. I need some kind of interface to enter a project name, and I haven't had great luck using TkInter yet.

Note: I've heard good things about PyQt for an interface

• I don't necessarily want those sub-directories for every project

VBScript

My best solution so far was to write a VBScript (.vbs format) to prompt me for a project name AND ask if I want to add those common project sub-folders.

Pros:

• Most efficient method so far
• Quick, easy, no renaming

Cons:

• Slightly longer code (24 lines without comments & line continuation), but it does include some more logic handling. View the code here.
• This is still generalized, and the "common subdirectories" option is hard coded (i.e. I don't need a Shapefiles directory in a video editing project).

Overall, This VBScript method opens the door to writing a slightly larger executable in Visual Studio with a more dynamic interface to add any kind of project folder - but that moves away from the original requirement of a highly rapid way to add a project folder automatically. I think this nails it.

Repair Lat/Long Coordinate Errors on the Fly

Every once in a while, someone misses a step in processing raw lat/long coordinates. Sometimes data was transferred incorrectly, sometimes the data wasn't available.

Here's some data I threw together as an example - these are my favorite caving diving sites in Florida:

ID	Name	Latitude	Longitude
1	Jackson Blue	30.790475	-85.140083
2	Madison Blue	30.480518	83.244421
3	Orange Grove	30.127215	-83.13053
4	Peacock	30.122503	-83.1324
5	Little River	29.996677	-82.966328
6	Devil's Eye	29.83509	-82.69659
7	Devil's Ear	29.835297	-82.696568
8	Ginnie Spring	29.836178	-82.700148

Initial projection of data is missing a location

Something's wrong. I expected to see Madison Blue just north of that group of three points. (some of those spots are springs that are within a few hundred yards of each other.

! When projecting data, immediately zoom to that layer, or hit the Full Extent button  to ensure that all points were plotted properly.

ArcMap's Zoom to Full Extent tool

After I zoomed to the full extent of the data, I realized what must be wrong with the data (see below). Obviously there's a Longitude value that's missing its negative sign because I haven't been diving in India/Tibet before.

Zoom to full extent to check for mis-projected data.

A few quick tips:

• Locations on the same latitude, but on the other side of the world are probably missing or adding a negative sign
• Points off the coast of Nigeria/Ghana/Gabon, down under of Africa's west coast are probably inheriting a Null or erroneous value, so they're defaulting to the coordinate (0, 0).

• After zooming to the full extent of a map, use the Select Features tool to select those incorrect locations. In the Attribute Table, choose Show selected Records to take a look at the incorrect data.

Fix the Data

1. Start an Edit Session on that layer in question
2. Select the erroneous site. Mine is ID = 2, or Name = Madison Blue
3. From the Editor toolbar, choose the Sketch Properties button to open the Edit Sketch Properties window
4. Add a negative in front of the selected X value and press Enter or click Finish Sketch to apply the update

5. Save and Stop Editing

That's it! Piece of cake..

Corrected data plot

Color Development: Digital Color Basics

I intended to just write about how to create colors and suggest some cool tools I've found, but this got out of control. Check out the next article:

Color Development: Tools and Methods

There are various ways of building colors for a project or symbol. For general, rapid selection, users are often offered a basic color palette to select a color:

Default color palette from ArcMap

Color palette from Word

Even MS Paint offers a few basic colors

Notice that each of these provide an option to edit the color further.

Color Values

Colors commonly have names: like Black, White, Red, Orange, Yellow, Green, Blue, Purple, Cyan, Magenta, Olive Drab, etc.

! Specifics aside, Black, White, and shades of grays are referred to as colors here

Various environments like CSS, and HTML accept some reserved names to act as color values, however those names correspond to very specific numeric values (discussed below), and can be easily referenced on lists like this one. A color isn't "like red" to a computer. It's Red, or it's not.

In the world of coding, there's a need to describe color in a more articulate way than just names. Due to the diverse nature of how light acts (See Further Reading, at the end of this article), there are also a number of different methods that programmers and graphic artists use to build and digitally represent color. Although the various methods may describe a color value differently, they are basically just providing a group of three or more numerical values that act as measurements of various criteria that go into building a representation of color. There are four types of units you may expect to see in these groupings:

• Percentage - values from 0 to 100%
• 8-bit - Values from 0 to 255
• Hex - Values from 00 to FF
• As in 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F, 11, 12, etc.
• Degrees - Values from 0 to 360º

Zero will always equal zero in any of the first three modes, but the "full" value values will be either 100%, 255, or FF. Those three are all equal values, written in different base units; in the same way that a measurement of 10 centimeters  is 100% of 3.93701 inches.

I'll get to degrees later in HSV/HSL.

Color Models

RGB

RGB Selector in ArcMap

The RGB triad stands for Red, Green, and Blue. Colors in this model are altered by adding or subtracting values in for any of the three primary color channels. As an "additive model," greater values result in brighter color instances, and lower values result in darker colors. This is based on a model of emitted light, where mixing red, green, and blue light produce the color white.

Users can adjust the sliders in this interface to control values of each of these channels between values of 0 and 255; where 0 = no value, and 255 = the full value of a given channel.

Alternatively, users can type a number they have in mind - this is particularly helpful when one wants to jump between increments of a values when searching for a color to use (e.g. changing the red value from 10, to 20, to 30, using the keyboard's number pad).

Color values are read as (#, #, #) or 

(Red value, Green value, Blue value). Since the RGB model is 24-bit (16.7 million possible color combinations), the values in each of the three channels will be some combination between 0 and 255.

As the example above illustrates, to build the color Red, tune the Red channel all the way up to 255, and leave Green and Blue channels at 0 since there should be no other color in Red but red. Done. Aside from various settings on your personal monitor, this is the standard, globally accepted, fullest possible value of Red.

Mixing two of the primary colors, Red, Green, or Blue produces
the "additive secondary" colors, Cyan, Magenta, and Yellow.
Mixing all three primary colors produces White.

The value (0, 0, 0) produces Black (the absence of light), and (255, 255, 255) produces White. Shades of gray are produced by matching all three values from (1, 1, 1) (darkest gray but not Black) to (254, 254. 254) (lightest gray but not White).

Back to our example of building the color Red, adding some value to the Green or Blue channels will yield some other color that may look reddish, but it isn't Red.

Additonally, the ArcMap interface gives you hints as to what other colors can be created from the current settings. With Red at 255, adjusting the Green channel will produces shades of red, through orange, and finally arriving at Yellow. Alternatively, adjusting the Blue channel will move through shades of reds and pinks and finally arrive at Magenta.

This is a good model to use to begin learning how to mix colors, but it's not my preferred model to use because it's tricky to imagine how to add a little red or remove a little green to target your preferred color.

CMYK

CMYK is a subtractive color model, used most often when printing is involved as an output medium. Pigments and inks absorb light (as opposed to how RGB works to emulate emitted light), so this model helps printers create the correct color with their ink print heads, as opposed to a computer monitor's LED pixels.

CMYK stands for Cyan-Magenta-Yellow-Key, where Key represents a black ink head. Black is used in printers as an efficient and controlled way to provide shadows, dark values, and black text to the printed material.

Just like mixing color with paint (where light is absorbed and the remaining wavelengths provide the observer perceived color), the CMYK model begins with the color white. Value is added to the channels, and lightness is removed. So zero values are White, and full values in all channels produce Black. Additionally, a full value in the Key channel also produces Black, regardless of the settings in the CMY channels.

Notice that the secondary colors in the RGB model produce Cyan, Magenta, and Yellow. Conversely, mixing two primary colors in the CMYK model will produce Red, Green, and Blue.

This is also not my favorite model to use to produce color, and it's generally just a pain in my ass when I'm printing large format maps.

HSV / HSL / HSVa / HSLa

HSV dialog in ArcMap

HSV selectors in Gimp, adjacent to RGB selectors
with the Hex code value, below

HSLa Selectors in Inkscape

These triads stand for Hue-Saturation-Value, and  Hue-Saturation-Lightness. The following "a" stands for Alpha, which is a metric of transparency. Sometimes this is included as a channel, sometimes it isn't. It's very important to recognize whether or not it's included. If you don't want to bother, make sure it is set to 100%, or 255, or FF.

This model is a bit tricky, so let's break it down.

Hue can be though of as the root color you're looking for on the color wheel: either a reddish, or a blueish, or a yellowish color, etc. The wheel starts at 0º, and continues through oranges, yellows, greens, blues etc, and eventually ends back at red at 360º; which also = 0º, but that's okay because both values are the same red.

A color wheel with 0° and 360° at the 12:00 position

Next, Saturation describes how muted gray (lack of color) or vibrant (fully colored) a color should be. This is usually a percentage, where 0 means no color is present and 100% represents the full colored potential of a color.

Examples of saturation gradients for several hues,
which correspond to the color wheel, above

! Note that the units can change between software packages. Inkscape uses 8-bit values (numbers 0-255) for each, HSLa channel. So a Hue value of 255 will correlate to 360° on the color wheel, and a 255 in the Saturation channel is equivalent to 100% saturation.

If you are building colors that are way off, you may need to check your units and convert when necessary.

Finally, Value and Lightness and are.. Also a bit tricky. Go ahead and read about the differences over on Wikipedia. Essentially, this component controls how light or dark a color will be.

Zero values in both HSV and HSL produce the color Black, regardless of Hue or Saturation settings.

The main difference between Value and Lightness is that the highest Value setting produces a fully saturated color (see the Saturation illustration above, and the ArcMap HSV selector below), and

the highest Lightness setting will produce a pure White color (see the Inkscape HSL selector below). A fully saturated color using HSL will have a Lightness value of 50% - or 128 (as Inkscape uses 8-bit values for each channel).

Hex

I didn't really touch on Hex, other than that it is another method of preserving a value of color. There's not much to it. It essentially is just a concatenation of the three or more channels in the fewest logical number of characters.

Short answer: Hexadecimal notation throws a couple of letters after the number 9 so each significant figure can be one of 16 possible values:

0 1 2 3 4 5 6 7 8  9  A  B  C  D  E  F

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Hex usually stands for RRGGBB, and sometimes for RRGGBBAA (for Alpha).

To find the Hex value of Red using the common six-digit RRGGBB method, first find the RGB value of Red:

  (255, 0, 0)

Then convert each value to its hex equivalent:

(FF, 00, 00)

Finally, remove the formatting, and stick the three values together:

FF0000
123456

For Inkscape and the few other examples of hex RGBa values, just throw another set of F's on the end. Here, we'll have a fully opaque value of Red:

FF0000FF

12345678

HTML & CSS

There are two ways to write hex color values in CSS and HTML.

There's the long way (again, using Red as an example):

#FF0000

And when all three RGB channels use doubled digits for their channel, you can use a 3-digit format:

#F00

Additional examples:

#112233 = #123

#00FF00 = #0F0

#333333 = #333

Further Reading

The physics of light are fascinating. I learned a lot about the subject in my various Remote Sensing courses in grad school, and in the practice of GIS and graphic design over the years. There are plenty of other great resources to use on these matters. Here are a few search terms to consider:

• Emitted vs. Reflected light
• Additive vs. Subtractive color
• Color models
• Primary color of light and pigment
• Light vs. Pigments
• Pigment vs. Dye
• Blackbody radiators
• Color theory

Add a Bing Maps Key for ArcMap

I was provided a key to use Bing Maps services in ArcGIS Desktop.  Here's how to set it up.

Enter a Bing Key

• Navigate to the bin directory where you installed ArcMap. Something like:

[Drive]:\Program Files (x86)\ArcGIS\Desktop10.2\bin

• Open the SetBingKey.exe program

• Paste your custom key into the text box and click OK

Load Bing Maps Layers Into a Project

Now you can add Bing layers in ArcMap:

• Click the Add Data dropdown and choose Add Data from ArcGIS Online… 

• Type Bing as a search term and hit Enter
• Click Add on any of the Bing Maps basemaps you want to use

* For versions other than 10.2, read more from Esri's Knowledge Base

3D Flight Data Visualization

I recently came across some delicious raw data from a flight tracking tool. Here's what it looks like in 3D!

Grab Flight Data

• Go to FlightAware.com
• Search for a flight and select it to . I found a flight from Minneapolis to New York City - a flight I'd like to be riding right now
• Click the track log & graph on the right information panel 

• Open Excel, Google Sheets, etc.
• Build the field headings separately since they don't select well with the data values from the Web page
• Select the data from the page, and copy it

Copy the data by itself - Build/copy headings separately

• Paste Special into your spreadsheet with text/values only - no formatting! If you directly paste the selection, there may be a bunch of graphic elements that migrate over too

Format & Convert

• Fill in the gaps of any altitude values that aren't available. I used the Fill Down (Ctrl+D) tool to duplicate the previous reading

• Make a new field called Linestring
• Use this formula to concatenate Longitude, Latitude, and Altitude together, separated with commas. Fill this down for the entire dataset. We'll come back to this in a moment

=C2 & "," & B2 & "," & H2

• Copy the following KML template and paste it into your favorite text editor (Notepad, Notepad++, Sublime Text, etc.)

<?xml version="1.0" encoding="UTF-8"?><br /> <kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2" xmlns:kml="http://www.opengis.net/kml/2.2" xmlns:atom="http://www.w3.org/2005/Atom"><br /> <document><br />     <name>Flight Visualization</name><br />         <placemark><br />             <name>Flight Path</name><br />             <linestring><br />                 <coordinates><br />                     <!-- Replace this line with Lat,Long,Altitude values --><br />                 </coordinates><br />             </LineString><br />         </Placemark><br /> </Document><br /> </kml>

• Replace the comment on line 9 (<!-- Replace... -->, between the <coordinate> tags) with the data (not the field heading) of the new Linestring field
• Save this as Flight Visualization.kml, or something appropriate

Display in Google Earth

• Double click the new KML file to load in Google Earth
• Expand the new folder until you find the Flight Path. Right click on that Path layer and choose Properties

• Navigate to the Altitude Tab. Change the Altitude setting from Clamped to ground to Absolute. Make sure that Extend path to ground is checked
• On the Style, Color tab, choose a bright red Lines color. Use a Width of 1.0. Set a 30% Opacity for the Area symbol. I like the Filled + Outlined setting
• Play with these settings until you're happy. Click OK to start playing with your visualization