JPEG Archive - Utilities for archiving JPEGs for long term storage

[Arthur]

All four of the utilities are available for Windows, macOS and Linux, except for jpeg-archive which is macOS and Linux only since it's a bash script.

 

The jpeg-archive script simply runs jpeg-recompress with various parameters so it wouldn't be difficult to create an equivalent batch file or PowerShell script, or even run it via WSL in Windows 10.

 

Website / Download

 

Utilities for archiving photos for saving to long term storage or serving over the web. The goals are:

 

• Use a common, well supported format (JPEG)
  
• Minimize storage space & cost
  
• Identify duplicates / similar photos
  

Approach:

• Command-line utilities & scripts
  
• Simple options & useful help
  
• Good quality output via sane defaults
  

 

The following utilities are part of this project. All of them accept a --help parameter to see the available options.

 

jpeg-archive

Compress RAW and JPEG files in a folder utilizing all CPU cores. This is a simple bash script that uses the utilities below. It requires:

 

• Bash
  
• Ladon
  
• dcraw
  
• exiftool
  
• jpeg-recompress (part of this project)
  

# Compress a folder of images
cd path/to/photos
jpeg-archive

# Custom quality and metric
jpeg-archive --quality medium --method smallfry

 

jpeg-recompress

Compress JPEGs by re-encoding to the smallest JPEG quality while keeping perceived visual quality the same and by making sure huffman tables are optimized. This is a lossy operation, but the images are visually identical and it usually saves 30-70% of the size for JPEGs coming from a digital camera, particularly DSLRs. By default all EXIF/IPTC/XMP and color profile metadata is copied over, but this can be disabled to save more space if desired.

 

There is no need for the input file to be a JPEG. In fact, you can use jpeg-recompress as a replacement for cjpeg by using PPM input and the --ppm option.

 

The better the quality of the input image is, the better the output will be.

 

Some basic photo-related editing options are available, such as removing fisheye lens distortion.

 

# Default settings
jpeg-recompress image.jpg compressed.jpg

# High quality example settings
jpeg-recompress --quality high --min 60 image.jpg compressed.jpg

# Slow high quality settings (3-4x slower than above, slightly more accurate)
jpeg-recompress --accurate --quality high --min 60 image.jpg compressed.jpg

# Use SmallFry instead of SSIM
jpeg-recompress --method smallfry image.jpg compressed.jpg

# Use 4:4:4 sampling (disables subsampling).
jpeg-recompress --subsample disable image.jpg compressed.jpg

# Remove fisheye distortion (Tokina 10-17mm on APS-C @ 10mm)
jpeg-recompress --defish 2.6 --zoom 1.2 image.jpg defished.jpg

# Read from stdin and write to stdout with '-' as the filename
jpeg-recompress - - compressed.jpg

# Convert RAW to JPEG via PPM from stdin
dcraw -w -q 3 -c IMG_1234.CR2 | jpeg-recompress --ppm - compressed.jpg

# Disable progressive mode (not recommended)
jpeg-recompress --no-progressive image.jpg compressed.jpg

# Disable all output except for errors
jpeg-recompress --quiet image.jpg compressed.jpg

 

jpeg-compare

Compare two JPEG photos to judge how similar they are. The fast comparison method returns an integer from 0 to 99, where 0 is identical. PSNR, SSIM, and MS-SSIM return floats but require images to be the same dimensions.

 

# Do a fast compare of two images
jpeg-compare image1.jpg image2.jpg

# Calculate PSNR
jpeg-compare --method psnr image1.jpg image2.jpg

# Calculate SSIM
jpeg-compare --method ssim image1.jpg image2.jpg

 

jpeg-hash

Create a hash of an image that can be used to compare it to other images quickly.

 

jpeg-hash image.jpg