Checksum File (SUM)

Overview

SUM files are checksum files that contain hash values used to verify the integrity of other files. They store cryptographic checksums (typically MD5, SHA-1, or SHA-256) alongside filenames to enable data verification and corruption detection.

File Details

• Extension: .sum
• MIME Type: text/plain
• Category: Data
• Binary/Text: Text

Technical Specifications

Format Structure

SUM files typically follow one of these formats:

# BSD-style format
MD5 (filename.ext) = checksum_value

# GNU coreutils format
checksum_value  filename.ext

# Multiple files
checksum1  file1.txt
checksum2  file2.txt
checksum3  file3.txt

Common Hash Algorithms

• MD5: 128-bit (32 hex characters)
• SHA-1: 160-bit (40 hex characters)
• SHA-256: 256-bit (64 hex characters)
• SHA-512: 512-bit (128 hex characters)

History

• 1970s: Early checksum algorithms developed
• 1991: MD5 algorithm published
• 1993: SHA-1 introduced by NSA
• 2001: SHA-256 and SHA-512 standardized
• 2005: MD5 vulnerabilities discovered

Structure Details

File Format Variants

BSD Format

MD5 (archive.zip) = d41d8cd98f00b204e9800998ecf8427e
SHA256 (document.pdf) = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

GNU Format

d41d8cd98f00b204e9800998ecf8427e  archive.zip
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  document.pdf

Simple Format

file1.txt:5d41402abc4b2a76b9719d911017c592
file2.txt:098f6bcd4621d373cade4e832627b4f6

Code Examples

Creating Checksums (Python)

import hashlib
import os

def create_sum_file(directory, algorithm='sha256'):
    """Create a checksum file for all files in directory"""
    checksums = []
    
    for root, dirs, files in os.walk(directory):
        for file in files:
            filepath = os.path.join(root, file)
            relative_path = os.path.relpath(filepath, directory)
            
            # Calculate checksum
            hash_obj = hashlib.new(algorithm)
            with open(filepath, 'rb') as f:
                for chunk in iter(lambda: f.read(4096), b""):
                    hash_obj.update(chunk)
            
            checksum = hash_obj.hexdigest()
            checksums.append(f"{checksum}  {relative_path}")
    
    # Write sum file
    with open(f'checksums.{algorithm}', 'w') as f:
        f.write('\n'.join(checksums))

# Usage
create_sum_file('/path/to/files', 'sha256')

Verifying Checksums

def verify_checksums(sum_file):
    """Verify files against checksum file"""
    results = {'passed': 0, 'failed': 0, 'missing': 0}
    
    with open(sum_file, 'r') as f:
        for line in f:
            line = line.strip()
            if not line or line.startswith('#'):
                continue
                
            # Parse checksum and filename
            parts = line.split('  ', 1)
            if len(parts) != 2:
                continue
                
            expected_checksum, filename = parts
            
            if not os.path.exists(filename):
                print(f"MISSING: {filename}")
                results['missing'] += 1
                continue
            
            # Calculate actual checksum
            hash_obj = hashlib.sha256()
            with open(filename, 'rb') as f:
                for chunk in iter(lambda: f.read(4096), b""):
                    hash_obj.update(chunk)
            
            actual_checksum = hash_obj.hexdigest()
            
            if actual_checksum == expected_checksum:
                print(f"OK: {filename}")
                results['passed'] += 1
            else:
                print(f"FAILED: {filename}")
                results['failed'] += 1
    
    return results

Bash Script Generation

#!/bin/bash
# Generate MD5 checksums
find . -type f -name "*.txt" -exec md5sum {} \; > checksums.md5

# Verify checksums
md5sum -c checksums.md5

# Generate SHA-256 checksums
find . -type f -name "*.pdf" -exec sha256sum {} \; > documents.sha256

Tools and Applications

Command Line Tools

• md5sum/sha256sum: GNU coreutils
• md5/shasum: BSD utilities
• certUtil: Windows built-in
• hashdeep: Advanced hashing tool

GUI Applications

• HashTab: Windows shell extension
• QuickHash: Cross-platform GUI
• HashMyFiles: Windows utility
• GtkHash: Linux GTK application

Programming Libraries

• hashlib (Python): Built-in hashing
• crypto (Node.js): Cryptographic functions
• Digest (Ruby): Hash algorithms
• MessageDigest (Java): Hash computations

Best Practices

File Organization

• Use descriptive filenames for sum files
• Include algorithm type in filename
• Group related files logically
• Document checksum purposes

Security Considerations

• Use strong hash algorithms (SHA-256+)
• Avoid MD5 for security-critical applications
• Store checksums separately from data
• Implement digital signatures for authenticity

Automation

• Generate checksums automatically
• Include in CI/CD pipelines
• Schedule regular verification
• Monitor for failures

Security Considerations

Hash Algorithm Selection

• MD5: Fast but cryptographically broken
• SHA-1: Deprecated for security use
• SHA-256: Current standard
• SHA-3: Future-proof option

Attack Vectors

• Collision attacks: Generate same hash
• Rainbow tables: Precomputed hashes
• Birthday attacks: Exploit probability
• Length extension: Algorithm-specific

Protection Measures

• Use salt for sensitive data
• Implement HMAC for authentication
• Store checksums securely
• Regular algorithm updates

Common Use Cases

Data Integrity

• File transfer verification
• Archive validation
• Backup verification
• Storage corruption detection

Digital Forensics

• Evidence integrity
• Chain of custody
• Incident response
• Malware analysis

Software Distribution

• Package verification
• Update integrity
• License compliance
• Tamper detection

File Format Variations

Extended Formats

# With timestamps
checksum_value  filename.ext  2024-01-15T10:30:00Z

# With file sizes
checksum_value  filename.ext  1024

# With multiple algorithms
MD5:checksum1,SHA1:checksum2  filename.ext

Metadata Inclusion

• File modification times
• File sizes
• Permission bits
• Extended attributes

Checksum files are essential for data integrity verification, providing a simple yet effective method to detect file corruption, unauthorized modifications, and transmission errors across various computing environments.