Skip to main content

File Diff

Overview​

File Diff enables you to compare data from different file formats, making it easy to validate data exports, ETL processes, or data migrations.

Supported file formats include CSV, Excel, JSON, XML, and other common file formats.

File Diff is one of three workflow types in the SmartDiff editor — the others being Data Source Diff and Query Diff. You choose the type once, at the top of the editor (the File tab in the type tabs). File workflows use the same four-tab flow as Query Diff — Mapping, Configuration, Validation, and Results; Filters and Transforms are only available on Data Source Diff workflows.

Supported File Formats​

  • CSV/TSV: Comma/Tab-separated values
  • Parquet/Avro/ORC: Columnar storage file formats
  • Excel: .xlsx and .xls formats
  • JSON: JavaScript Object Notation
  • XML: eXtensible Markup Language
  • Fixed Width: Fixed-width text files
  • Custom Delimited: User-specified delimiters

Key Features​

  • Multiple Format Support: Compare files in different formats
  • File Pairs: Add multiple source/target file pairs to one workflow, each analyzed and mapped independently
  • Schema Mapping: Columns are auto-mapped by name after Analyze; unmapped columns get fuzzy-match suggestions
  • Large File Handling: Efficient processing of large files
  • Character Encoding Support: Automatic detection and handling of different encodings

Advanced Options​

  • Ignore header rows
  • Case-sensitive comparison
  • Trim whitespace
  • Handle null/empty values
  • Custom date/time formats

File-Specific Guides​

  • CSV File Diff – Step-by-step guide for comparing CSV (and other table-structured) files
  • XML File Diff – Step-by-step guide for comparing XML (and other hierarchical) files

Best Practices​

tip
  • Use consistent file encodings (UTF-8 recommended)
  • Include headers in your files for better column identification
  • For large files, consider splitting them into smaller chunks
  • Workflows auto-save as you edit, so a recurring validation just needs Run diff again — or Duplicate workflow from the Workflows list as a starting point for a similar comparison
  • Review the summary statistics before analyzing detailed differences

Troubleshooting​

warning
  • File Format Issues: Ensure files are not corrupted and match the specified format
  • Encoding Problems: If you see special character issues, verify the file encoding
  • Column Mismatches: Verify that column names and data types match between files