Skip to content

Storage System

Solana Insider Monitor includes a robust storage system that persists wallet data, tracks historical changes, and enables advanced analysis of wallet activities over time.

Overview

The storage system ensures that historical wallet data is preserved across application restarts and provides a foundation for trend analysis, change detection, and alert validation.

Storage Options

Solana Insider Monitor offers multiple storage options to fit different use cases:

JSON File Storage

The default storage method uses JSON files to persist wallet data:

  • Data file: Stores current wallet balances and token information
  • History file: Maintains a record of balance changes over time
  • Alert file: Stores alerts generated by the system

This approach is suitable for most users and requires no additional setup.

Memory-Only Mode

For testing or temporary monitoring, you can run the monitor in memory-only mode with the -no-storage flag:

./bin/monitor -no-storage

In this mode, data is not persisted between application restarts.

Data Structure

Wallet Data

The storage system maintains the following information for each wallet:

  • Wallet address
  • Current token balances
  • Token metadata (name, symbol, decimals)
  • Last update timestamp
  • Associated labels or tags

Historical Data

Historical data includes:

  • Token balance snapshots over time
  • Percentage and absolute changes
  • USD value estimation (when available)
  • Timestamps for each data point

Data Management

Data Retention

By default, historical data is retained for 30 days. You can customize the retention period in the configuration:

"storage": {
  "history_retention_days": 60
}

Data Backup

It's recommended to regularly back up the data directory to prevent data loss. You can specify a custom data directory location:

./bin/monitor -data-dir /path/to/backup/location

Data Import/Export

The storage system supports importing and exporting data:

  • Export: Save current data to a JSON file for backup or analysis
  • Import: Load data from a previously exported file

These operations can be performed via the web interface or command-line flags.

Performance Considerations

File Size Management

For long-running instances monitoring many wallets, the history file can grow large. The system automatically performs compaction to manage file size:

"storage": {
  "auto_compact": true,
  "compact_threshold_mb": 100
}

Data Consistency

The storage system ensures data consistency through:

  • Atomic write operations
  • Regular checkpoints
  • Validation on read/write

Advanced Features

Data Analysis

The stored historical data enables several analytical capabilities:

  • Balance Trends: Analyze how wallet balances change over time
  • Volatility Detection: Identify wallets with frequent large changes
  • Pattern Recognition: Detect repeated balance change patterns

Custom Queries

When using the web interface, you can perform custom queries on the stored data:

  • Filter by date range
  • Group by token type
  • Sort by change magnitude
  • Export results as CSV

Data Privacy

The storage system does not collect any personal information beyond what's available on the public blockchain. All data stored is derived from public on-chain activity.

Troubleshooting

If you encounter issues with the storage system:

  1. Check if the data directory is writable
  2. Verify disk space is sufficient
  3. Consider compacting the history file if it's grown large
  4. Check for file corruption (the system will attempt auto-recovery)

For more help, see the Troubleshooting guide.

Future Enhancements

Planned improvements to the storage system include:

  • Database backend options (SQLite, PostgreSQL)
  • Cloud storage integration
  • Enhanced data compression
  • More advanced query capabilities