TAR (Tape Archive)

Overview

TAR (Tape Archive) is an archive file format used to collect multiple files and directories into a single file while preserving file system information such as user and group permissions, access and modification dates, and directory structures. Originally designed for sequential tape storage, TAR has become a standard archiving format in Unix and Linux systems.

Technical Details

File Structure

• Header Block: 512-byte header for each file/directory
• Data Blocks: File content in 512-byte blocks
• Padding: Zero-filled blocks to align data
• End Markers: Two consecutive zero-filled blocks mark end

Header Format (POSIX.1-1988)

Field Name       Offset  Size   Description
name             0       100    File name
mode             100     8      File permissions (octal)
uid              108     8      User ID (octal)
gid              116     8      Group ID (octal)
size             124     12     File size (octal)
mtime            136     12     Modification time (octal)
chksum           148     8      Header checksum (octal)
typeflag         156     1      File type indicator
linkname         157     100    Link target name
magic            257     6      "ustar\0" for POSIX format
version          263     2      "00" for POSIX
uname            265     32     User name
gname            297     32     Group name
devmajor         329     8      Device major number
devminor         337     8      Device minor number
prefix           345     155    Path prefix

File Type Indicators

• '0' or NUL: Regular file
• '1': Hard link
• '2': Symbolic link
• '3': Character special device
• '4': Block special device
• '5': Directory
• '6': FIFO (named pipe)
• '7': Contiguous file

History and Development

TAR was first introduced in Version 7 Unix in 1979 by John Gilmore. It was originally designed for backing up files to magnetic tape drives, hence the name "Tape Archive." The format has evolved through several standards including POSIX.1-1988 (ustar), POSIX.1-2001 (pax), and GNU extensions, each adding new capabilities while maintaining backward compatibility.

Code Examples

Basic TAR Operations

# Create archive
tar -cf archive.tar file1.txt file2.txt directory/

# Create compressed archive (gzip)
tar -czf archive.tar.gz directory/

# Create compressed archive (bzip2)
tar -cjf archive.tar.bz2 directory/

# Create compressed archive (xz)
tar -cJf archive.tar.xz directory/

# Extract archive
tar -xf archive.tar

# Extract to specific directory
tar -xf archive.tar -C /destination/path

# List archive contents
tar -tf archive.tar

# Verbose operations
tar -cvf archive.tar directory/  # Create with verbose output
tar -xvf archive.tar             # Extract with verbose output

# Exclude files during creation
tar -cf archive.tar --exclude="*.log" --exclude="temp/" directory/

# Update archive (add newer files)
tar -uf archive.tar newfile.txt

# Append to existing archive
tar -rf archive.tar additional_file.txt

Python TAR Manipulation

import tarfile
import os
from datetime import datetime

class TarManager:
    @staticmethod
    def create_archive(archive_name, source_paths, compression=None):
        """Create TAR archive from files/directories"""
        mode = 'w'
        if compression == 'gzip':
            mode = 'w:gz'
        elif compression == 'bzip2':
            mode = 'w:bz2'
        elif compression == 'xz':
            mode = 'w:xz'
        
        with tarfile.open(archive_name, mode) as tar:
            for path in source_paths:
                if os.path.exists(path):
                    tar.add(path, arcname=os.path.basename(path))
                    print(f"Added: {path}")
    
    @staticmethod
    def extract_archive(archive_name, extract_path='.', specific_files=None):
        """Extract TAR archive"""
        with tarfile.open(archive_name, 'r:*') as tar:
            if specific_files:
                # Extract specific files
                for file_name in specific_files:
                    try:
                        tar.extract(file_name, extract_path)
                        print(f"Extracted: {file_name}")
                    except KeyError:
                        print(f"File not found: {file_name}")
            else:
                # Extract all files
                tar.extractall(extract_path)
                print("Extracted all files")
    
    @staticmethod
    def list_archive_contents(archive_name):
        """List contents of TAR archive"""
        contents = []
        with tarfile.open(archive_name, 'r:*') as tar:
            for member in tar.getmembers():
                info = {
                    'name': member.name,
                    'size': member.size,
                    'mode': oct(member.mode),
                    'modified': datetime.fromtimestamp(member.mtime),
                    'type': 'file' if member.isfile() else 'directory' if member.isdir() else 'other',
                    'owner': f"{member.uname}:{member.gname}"
                }
                contents.append(info)
        return contents
    
    @staticmethod
    def add_file_to_archive(archive_name, file_path, arcname=None):
        """Add file to existing archive"""
        if arcname is None:
            arcname = os.path.basename(file_path)
        
        # Create temporary archive with new file
        temp_archive = archive_name + '.tmp'
        
        with tarfile.open(archive_name, 'r:*') as old_tar:
            with tarfile.open(temp_archive, 'w:gz') as new_tar:
                # Copy existing files
                for member in old_tar.getmembers():
                    if member.name != arcname:  # Avoid duplicates
                        new_tar.addfile(member, old_tar.extractfile(member))
                
                # Add new file
                new_tar.add(file_path, arcname=arcname)
        
        # Replace original archive
        os.replace(temp_archive, archive_name)
    
    @staticmethod
    def search_in_archive(archive_name, pattern):
        """Search for files matching pattern in archive"""
        import fnmatch
        matches = []
        
        with tarfile.open(archive_name, 'r:*') as tar:
            for member in tar.getmembers():
                if fnmatch.fnmatch(member.name, pattern):
                    matches.append(member.name)
        
        return matches

# Usage examples
tar_manager = TarManager()

# Create archive with compression
tar_manager.create_archive('backup.tar.gz', 
                          ['/home/user/documents', '/home/user/pictures'], 
                          compression='gzip')

# List contents
contents = tar_manager.list_archive_contents('backup.tar.gz')
for item in contents:
    print(f"{item['name']} - {item['size']} bytes - {item['modified']}")

# Extract specific files
tar_manager.extract_archive('backup.tar.gz', 
                           extract_path='/tmp/restore',
                           specific_files=['documents/important.txt'])

# Search for files
matches = tar_manager.search_in_archive('backup.tar.gz', '*.pdf')
print(f"PDF files found: {matches}")

C TAR Header Parsing

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>

#define TAR_BLOCK_SIZE 512

struct tar_header {
    char name[100];
    char mode[8];
    char uid[8];
    char gid[8];
    char size[12];
    char mtime[12];
    char chksum[8];
    char typeflag;
    char linkname[100];
    char magic[6];
    char version[2];
    char uname[32];
    char gname[32];
    char devmajor[8];
    char devminor[8];
    char prefix[155];
    char padding[12];
};

unsigned int calculate_checksum(struct tar_header *header) {
    unsigned int checksum = 0;
    unsigned char *bytes = (unsigned char *)header;
    
    // Calculate checksum treating chksum field as spaces
    for (int i = 0; i < TAR_BLOCK_SIZE; i++) {
        if (i >= 148 && i < 156) {
            checksum += ' ';  // chksum field treated as spaces
        } else {
            checksum += bytes[i];
        }
    }
    
    return checksum;
}

long octal_to_decimal(const char *octal_str, int length) {
    long result = 0;
    for (int i = 0; i < length && octal_str[i] != '\0' && octal_str[i] != ' '; i++) {
        if (octal_str[i] >= '0' && octal_str[i] <= '7') {
            result = result * 8 + (octal_str[i] - '0');
        }
    }
    return result;
}

int read_tar_archive(const char *filename) {
    FILE *file = fopen(filename, "rb");
    if (!file) {
        perror("Error opening file");
        return -1;
    }
    
    struct tar_header header;
    
    while (fread(&header, sizeof(header), 1, file) == 1) {
        // Check for end of archive (two zero blocks)
        if (header.name[0] == '\0') {
            break;
        }
        
        // Verify magic number for POSIX tar
        if (strncmp(header.magic, "ustar", 5) == 0) {
            printf("POSIX TAR format detected\n");
        }
        
        // Extract file information
        long file_size = octal_to_decimal(header.size, 12);
        long file_mode = octal_to_decimal(header.mode, 8);
        long mod_time = octal_to_decimal(header.mtime, 12);
        
        printf("File: %s\n", header.name);
        printf("  Size: %ld bytes\n", file_size);
        printf("  Mode: %lo\n", file_mode);
        printf("  Type: ");
        
        switch (header.typeflag) {
            case '0': case '\0': printf("Regular file\n"); break;
            case '1': printf("Hard link\n"); break;
            case '2': printf("Symbolic link\n"); break;
            case '5': printf("Directory\n"); break;
            default: printf("Other (%c)\n", header.typeflag); break;
        }
        
        // Skip file data blocks
        long blocks_to_skip = (file_size + TAR_BLOCK_SIZE - 1) / TAR_BLOCK_SIZE;
        fseek(file, blocks_to_skip * TAR_BLOCK_SIZE, SEEK_CUR);
    }
    
    fclose(file);
    return 0;
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        printf("Usage: %s <tar_file>\n", argv[0]);
        return 1;
    }
    
    return read_tar_archive(argv[1]);
}

Common Use Cases

System Backup and Archival

• Full System Backups: Complete filesystem snapshots
• Incremental Backups: Changed files since last backup
• Home Directory Archives: User data preservation
• Configuration Backups: System and application settings

Software Distribution

• Source Code Packages: Software distribution format
• Linux Packages: Base format for many package managers
• Documentation Archives: Manual and help file collections
• Asset Bundles: Game and application resources

Data Transfer and Migration

• Server Migration: Moving data between systems
• Cloud Uploads: Efficient bulk data transfer
• Network Transport: Preserving file attributes over networks
• Version Control: Git and other VCS backends

Development Workflows

• Build Artifacts: Compiled software packaging
• Deployment Packages: Application release bundles
• Container Images: Docker layer storage
• Patch Distribution: Software update packages

Tools and Utilities

Standard Unix Tools

• tar: Universal TAR implementation
• gtar (GNU tar): Extended GNU implementation
• bsdtar: BSD implementation with libarchive
• star: Jörg Schilling's TAR implementation

Compression Integration

• gzip (.tar.gz/.tgz): Most common compression
• bzip2 (.tar.bz2): Better compression ratio
• xz (.tar.xz): Modern high-compression format
• compress (.tar.Z): Legacy compression format

GUI Applications

• File Roller: GNOME archive manager
• Ark: KDE archive utility
• 7-Zip: Cross-platform archive manager
• WinRAR: Windows commercial archiver

Programming Libraries

• libarchive: C library for multiple archive formats
• Python tarfile: Built-in Python module
• Java Archive API: Java TAR support libraries
• Go archive/tar: Go standard library package

Advanced Features

GNU TAR Extensions

# Incremental backups
tar -c -g snapshot.snar -f backup-level0.tar /home
tar -c -g snapshot.snar -f backup-level1.tar /home  # Only changes

# Sparse file handling
tar -cSf archive.tar sparse_file.img

# Extended attributes and ACLs
tar -c --xattrs --acls -f archive.tar directory/

# Exclude patterns from file
tar -c -X exclude_patterns.txt -f archive.tar directory/

# Multi-volume archives
tar -c -M -L 1024 -f archive.tar directory/  # 1024KB volumes

Performance Optimization

• Parallel Compression: Using pigz instead of gzip
• Buffer Tuning: Adjusting block size for tape drives
• Network Optimization: ssh compression with TAR streams
• Memory Usage: Streaming for large archives

TAR remains one of the most important archiving formats in Unix-like systems, providing reliable file preservation and cross-platform compatibility for backup, distribution, and data transfer applications.