Text Protocol Buffer (textproto)

Overview

Text Protocol Buffer (textproto) is a human-readable text format for representing Protocol Buffer messages. It provides a way to serialize and deserialize protobuf data in a format that's easy to read, edit, and debug, making it ideal for configuration files and testing.

File Details

• Extension: .textproto, .txtpb, .pbtxt
• MIME Type: text/plain
• Category: Data
• Binary/Text: Text

Technical Specifications

Format Structure

Text protobuf uses a simple key-value syntax:

field_name: value
repeated_field: value1
repeated_field: value2
message_field {
  nested_field: value
}

Data Types Support

• Scalar types: int32, int64, float, double, bool, string
• Repeated fields: Lists of values
• Nested messages: Hierarchical structures
• Enums: Named constants
• Maps: Key-value pairs

History

• 2008: Protocol Buffers open-sourced by Google
• 2010: Text format specification formalized
• 2016: Proto3 introduces simplified syntax
• 2019: JSON mapping standardized
• 2020+: Continued evolution with new features

Structure Details

Basic Syntax

# Comments start with #
string_field: "Hello, World!"
int_field: 42
float_field: 3.14159
bool_field: true
bytes_field: "SGVsbG8="  # Base64 encoded

Repeated Fields

# Multiple values for repeated fields
tags: "important"
tags: "urgent"
tags: "work"

# Or array-like syntax
numbers: [1, 2, 3, 4, 5]

Nested Messages

person {
  name: "John Doe"
  age: 30
  address {
    street: "123 Main St"
    city: "Anytown"
    state: "CA"
    zip_code: "12345"
  }
}

Code Examples

Protocol Definition (.proto)

syntax = "proto3";

package example;

message Person {
  string name = 1;
  int32 age = 2;
  repeated string email = 3;
  
  message Address {
    string street = 1;
    string city = 2;
    string state = 3;
    string zip_code = 4;
  }
  
  Address address = 4;
  
  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }
  
  message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
  }
  
  repeated PhoneNumber phone = 5;
}

message AddressBook {
  repeated Person person = 1;
}

Text Protobuf Example

# example.textproto
person {
  name: "Alice Smith"
  age: 28
  email: "[email protected]"
  email: "[email protected]"
  
  address {
    street: "456 Oak Avenue"
    city: "Springfield"
    state: "IL"
    zip_code: "62701"
  }
  
  phone {
    number: "+1-555-123-4567"
    type: MOBILE
  }
  
  phone {
    number: "+1-555-987-6543"
    type: WORK
  }
}

person {
  name: "Bob Johnson"
  age: 35
  email: "[email protected]"
  
  address {
    street: "789 Pine Street"
    city: "Portland"
    state: "OR"
    zip_code: "97201"
  }
  
  phone {
    number: "+1-555-555-5555"
    type: HOME
  }
}

Python Processing

import google.protobuf.text_format as text_format
from your_proto_pb2 import AddressBook

def read_textproto(filename):
    """Read text protobuf from file"""
    address_book = AddressBook()
    
    with open(filename, 'r') as f:
        text_format.Merge(f.read(), address_book)
    
    return address_book

def write_textproto(address_book, filename):
    """Write protobuf to text format"""
    with open(filename, 'w') as f:
        f.write(text_format.MessageToString(address_book))

def convert_binary_to_text(binary_file, text_file):
    """Convert binary protobuf to text format"""
    address_book = AddressBook()
    
    with open(binary_file, 'rb') as f:
        address_book.ParseFromString(f.read())
    
    with open(text_file, 'w') as f:
        f.write(text_format.MessageToString(address_book))

# Usage
address_book = read_textproto('data.textproto')
for person in address_book.person:
    print(f"Name: {person.name}, Age: {person.age}")

C++ Processing

#include <google/protobuf/text_format.h>
#include <google/protobuf/io/zero_copy_stream_impl.h>
#include <fstream>
#include "example.pb.h"

bool ReadTextProto(const std::string& filename, AddressBook* address_book) {
    std::ifstream file(filename);
    if (!file.is_open()) {
        return false;
    }
    
    google::protobuf::io::IstreamInputStream input(&file);
    return google::protobuf::TextFormat::Parse(&input, address_book);
}

bool WriteTextProto(const AddressBook& address_book, const std::string& filename) {
    std::ofstream file(filename);
    if (!file.is_open()) {
        return false;
    }
    
    google::protobuf::io::OstreamOutputStream output(&file);
    return google::protobuf::TextFormat::Print(address_book, &output);
}

Tools and Applications

Protocol Buffer Tools

• protoc: Official compiler with text format support
• buf: Modern protobuf tooling
• prototool: Protobuf development toolkit
• grpcurl: Testing gRPC services with textproto

IDEs and Editors

• VS Code: Protocol Buffer extensions
• IntelliJ IDEA: Protobuf plugin
• Vim: Protobuf syntax highlighting
• Emacs: Protobuf mode

Validation Tools

# Validate textproto file
protoc --encode=AddressBook example.proto < data.textproto > /dev/null

# Convert between formats
protoc --decode=AddressBook example.proto < data.bin > data.textproto
protoc --encode=AddressBook example.proto < data.textproto > data.bin

Best Practices

File Organization

• Use descriptive filenames
• Group related configurations
• Maintain consistent formatting
• Include documentation comments

Syntax Guidelines

• Use consistent indentation (2 spaces)
• Add comments for complex structures
• Validate against schema
• Use meaningful field names

Configuration Management

# config.textproto
server_config {
  host: "localhost"
  port: 8080
  max_connections: 1000
  
  # TLS configuration
  tls_config {
    cert_file: "/path/to/cert.pem"
    key_file: "/path/to/key.pem"
    ca_file: "/path/to/ca.pem"
  }
  
  # Feature flags
  features {
    enable_logging: true
    enable_metrics: true
    enable_tracing: false
  }
}

Security Considerations

Input Validation

def safe_parse_textproto(content, message_type):
    """Safely parse textproto with validation"""
    try:
        message = message_type()
        text_format.Merge(content, message)
        
        # Validate required fields
        if not message.IsInitialized():
            raise ValueError("Required fields missing")
        
        return message
    except text_format.ParseError as e:
        raise ValueError(f"Parse error: {e}")

Access Control

• Restrict file system access
• Validate field values
• Implement size limits
• Monitor parsing errors

Data Sanitization

• Escape special characters
• Validate string encodings
• Check numeric ranges
• Prevent injection attacks

Common Use Cases

Configuration Files

# Application configuration
app_config {
  database {
    host: "db.example.com"
    port: 5432
    name: "myapp"
    ssl_mode: "require"
  }
  
  cache {
    redis_url: "redis://cache.example.com:6379"
    ttl_seconds: 3600
  }
  
  logging {
    level: "INFO"
    format: "json"
    outputs: ["stdout", "file"]
  }
}

Test Data

# test_data.textproto
test_cases {
  name: "valid_user"
  input {
    user_id: "12345"
    email: "[email protected]"
  }
  expected_output {
    status: "success"
    user_created: true
  }
}

test_cases {
  name: "invalid_email"
  input {
    user_id: "12346"
    email: "invalid-email"
  }
  expected_output {
    status: "error"
    error_message: "Invalid email format"
  }
}

Machine Learning Models

# model_config.textproto
model {
  name: "image_classifier"
  version: "v1.2.0"
  
  layers {
    type: "conv2d"
    filters: 32
    kernel_size: [3, 3]
    activation: "relu"
  }
  
  layers {
    type: "max_pooling2d"
    pool_size: [2, 2]
  }
  
  training_config {
    learning_rate: 0.001
    batch_size: 32
    epochs: 100
  }
}

Text Protocol Buffers provide an excellent balance between human readability and structured data representation, making them ideal for configuration management, testing, and debugging in protobuf-based systems.