Outlook File Format

What is an Outlook file?

An Outlook file typically refers to a Personal Storage Table (PST) file, which is Microsoft Outlook's proprietary format for storing email messages, calendar events, contacts, tasks, and other mailbox data locally on a user's computer. PST files serve as personal data repositories that allow users to manage their email and personal information offline.

File Extensions

• .pst (Personal Storage Table)
• .ost (Offline Storage Table)
• .msg (Individual message files)

MIME Type

• application/vnd.ms-outlook

History and Development

PST files were introduced with early versions of Microsoft Outlook in the 1990s as a solution for storing email data locally. The format has evolved significantly over the years to accommodate growing storage needs and enhanced functionality.

Evolution Timeline

• 1997: Original PST format introduced with Outlook 97
• 2003: Enhanced PST format with increased size limits
• 2007: Unicode PST format supporting international characters
• 2010: Improved compression and performance optimizations
• 2013: Enhanced security and corruption resistance
• Present: Continued refinement with modern Outlook versions

Technical Specifications

PST File Structure

PST files use a complex internal database structure:

PST File Structure:
├── Header Block (512 bytes)
├── Allocation Map Pages
├── Node Database
│   ├── Internal Nodes
│   └── Leaf Nodes
├── Block Database
│   ├── Data Blocks
│   └── Extended Blocks
└── Message Store
    ├── Folders
    ├── Messages
    ├── Attachments
    └── Properties

Format Variants

• ANSI PST: Legacy format with 2GB size limit
• Unicode PST: Modern format supporting up to 50GB
• OST Files: Cached copies of server-based mailboxes

Internal Database Structure

Database Layers:
├── NDB Layer (Node Database)
│   ├── Heap Management
│   ├── B-tree Structure
│   └── Property Contexts
├── LTP Layer (Lists, Tables, Properties)
│   ├── Property Lists
│   ├── Table Contexts
│   └── Heap-on-Node
└── Messaging Layer
    ├── Folder Objects
    ├── Message Objects
    └── Attachment Objects

File Types and Purposes

PST Files (Personal Storage Table)

• Local Storage: Store mail data on local computer
• Archive Storage: Long-term email archiving
• Backup Purpose: Email data backup and migration
• Offline Access: Work with email without server connection

OST Files (Offline Storage Table)

• Cached Mode: Local cache of Exchange server mailbox
• Synchronization: Two-way sync with server
• Performance: Faster email access and searching
• Offline Work: Continue working without network connection

MSG Files (Individual Messages)

• Single Messages: Individual email messages
• Sharing: Send email files outside Outlook
• Archival: Store specific important messages
• Forensics: Email investigation and analysis

Data Storage and Organization

Folder Hierarchy

PST Folder Structure:
├── Personal Folders
│   ├── Inbox
│   ├── Outbox
│   ├── Sent Items
│   ├── Deleted Items
│   ├── Calendar
│   ├── Contacts
│   ├── Tasks
│   ├── Journal
│   ├── Notes
│   └── Custom Folders
└── Search Folders (Virtual)

Message Properties

PST files store comprehensive message data:

• Standard Properties: To, From, Subject, Date, Body
• Extended Properties: Message flags, categories, importance
• Custom Properties: User-defined fields and metadata
• Attachments: Embedded files and objects
• Digital Signatures: Security and authentication data

Data Types Stored

• Email Messages: Text and HTML email content
• Calendar Items: Appointments, meetings, events
• Contacts: Contact information and distribution lists
• Tasks: To-do items and task assignments
• Journal Entries: Activity tracking and notes
• Notes: Quick notes and reminders

Management and Operations

Creating PST Files

' VBA example to create PST file
Dim olApp As Outlook.Application
Dim olNamespace As Outlook.Namespace
Dim olFolder As Outlook.MAPIFolder

Set olApp = CreateObject("Outlook.Application")
Set olNamespace = olApp.GetNamespace("MAPI")

' Add PST file
olNamespace.AddStore "C:\Data\Archive.pst"

' Access the new PST
Set olFolder = olNamespace.Folders("Archive")

Opening and Accessing PST Files

# PowerShell script to access PST data
Add-Type -AssemblyName Microsoft.Office.Interop.Outlook

$outlook = New-Object -ComObject Outlook.Application
$namespace = $outlook.GetNamespace("MAPI")

# Open PST file
$pstFile = "C:\Data\mailbox.pst"
$namespace.AddStore($pstFile)

# Access folders
$pstRoot = $namespace.Stores | Where-Object {$_.FilePath -eq $pstFile}
$inbox = $pstRoot.GetRootFolder().Folders("Inbox")

# Process messages
foreach ($message in $inbox.Items) {
    Write-Host "Subject: $($message.Subject)"
    Write-Host "From: $($message.SenderName)"
    Write-Host "Date: $($message.ReceivedTime)"
    Write-Host "---"
}

Tools and Utilities

Microsoft Tools

• Outlook: Native PST file management
• SCANPST.EXE: PST file repair utility
• Import/Export Wizard: Data migration tools
• Archiving Assistant: Automated archiving setup

Third-party Tools

• PST Walker: PST file viewer and browser
• Kernel PST Viewer: Free PST examination tool
• SysTools PST Viewer: Commercial PST analysis software
• libpst: Open-source PST processing library

Forensic Tools

• EnCase: Enterprise forensic PST analysis
• FTK (Forensic Toolkit): Email investigation platform
• X-Ways Forensics: Advanced PST examination
• MailXaminer: Email forensic analysis tool

Programming Libraries

# Python example using pypff library
import pypff

# Open PST file
pst_file = pypff.file()
pst_file.open("mailbox.pst")

# Access folder tree
root_folder = pst_file.get_root_folder()

def process_folder(folder):
    print(f"Folder: {folder.get_name()}")
    
    # Process messages
    for message in folder.sub_messages:
        print(f"  Subject: {message.get_subject()}")
        print(f"  From: {message.get_sender_name()}")
        print(f"  Date: {message.get_delivery_time()}")
    
    # Process subfolders
    for subfolder in folder.sub_folders:
        process_folder(subfolder)

process_folder(root_folder)
pst_file.close()

Size Limitations and Management

Size Limits by Version

• Outlook 97-2002: 2GB maximum (ANSI format)
• Outlook 2003+: 20GB default, 50GB maximum (Unicode format)
• Configurable Limits: Registry settings for size warnings and limits

Size Management Strategies

# PowerShell script to check PST file sizes
$pstFiles = Get-ChildItem -Path "C:\Users\*\Documents\Outlook Files\*.pst" -Recurse

foreach ($pst in $pstFiles) {
    $sizeMB = [math]::Round($pst.Length / 1MB, 2)
    if ($sizeMB -gt 1000) {
        Write-Warning "Large PST file: $($pst.Name) - $sizeMB MB"
    } else {
        Write-Host "PST file: $($pst.Name) - $sizeMB MB"
    }
}

Performance Optimization

• Regular Compacting: Remove deleted item space
• Archiving: Move old items to archive PST files
• Folder Organization: Avoid oversized folders
• Index Maintenance: Rebuild search indexes when needed

Security and Encryption

Password Protection

' VBA to set PST password
Dim olApp As Outlook.Application
Dim olNamespace As Namespace
Dim olStore As Store

Set olApp = CreateObject("Outlook.Application")
Set olNamespace = olApp.GetNamespace("MAPI")

' Access PST store
Set olStore = olNamespace.Stores("Personal Folders")

' Change password (requires user interaction)
olStore.PropertyAccessor.SetProperty("http://schemas.microsoft.com/mapi/proptag/0x67010102", "newpassword")

Security Considerations

• Password Protection: Basic password-based encryption
• Access Control: File system permissions
• Data Residue: Deleted items may remain in file
• Backup Security: Secure storage of PST backups
• Network Transfer: Encrypt PST files during transfer

Migration and Conversion

PST to Other Formats

# Using libpst (Linux/Unix)
readpst -o output_directory mailbox.pst

# Convert to mbox format
readpst -r -j 4 -o mbox_output mailbox.pst

# Export to Thunderbird format
readpst -M -d thunderbird_output mailbox.pst

Exchange Migration

# PowerShell script for Exchange migration
Import-Module ExchangeOnlineManagement

# Import PST to Exchange Online
New-MailboxImportRequest -Mailbox [email protected] -FilePath "\\server\share\user.pst" -TargetRootFolder "Imported Mail"

# Monitor import progress
Get-MailboxImportRequest | Get-MailboxImportRequestStatistics

Troubleshooting and Repair

Common Issues

• Corruption: File corruption due to improper shutdown
• Size Limits: Reaching maximum file size limits
• Performance: Slow access with large files
• Synchronization: OST sync problems with Exchange

Repair Procedures

:: Run SCANPST.EXE for repair
"C:\Program Files\Microsoft Office\root\Office16\SCANPST.EXE" "C:\path\to\mailbox.pst"

:: Automated repair script
@echo off
set PST_PATH="C:\Users\%USERNAME%\Documents\Outlook Files\mailbox.pst"
set SCANPST="C:\Program Files\Microsoft Office\root\Office16\SCANPST.EXE"

echo Checking PST file integrity...
%SCANPST% %PST_PATH%

if %ERRORLEVEL% neq 0 (
    echo PST file repair may be needed
) else (
    echo PST file appears healthy
)

Prevention Strategies

• Regular Backups: Scheduled PST file backups
• Proper Shutdown: Close Outlook before system shutdown
• Size Monitoring: Track file size growth
• Regular Maintenance: Periodic compacting and archiving

Best Practices

File Management

1. Location: Store PST files on local drives, not network shares
2. Backup: Regular automated backups of PST files
3. Size Control: Monitor and manage file sizes
4. Organization: Use descriptive names and folder structures

Performance Guidelines

• Avoid Network Storage: Keep PST files on local drives
• Regular Compacting: Compress files to remove deleted space
• Folder Management: Avoid extremely large folders
• Archive Strategy: Move old data to archive PST files

Security Recommendations

• Password Protection: Use strong passwords for sensitive PST files
• Access Control: Implement proper file permissions
• Encryption: Encrypt PST files for sensitive data
• Secure Disposal: Securely delete old PST files

PST files remain a critical component of Microsoft Outlook's email management system, providing users with flexible local storage options while presenting unique challenges in terms of size management, security, and maintenance that require careful consideration in enterprise environments.