Skip to content

Frequently Asked Questions

General Questions

What is the MSM Data Collector?

The MSM Data Collector is a tool that extracts telemetry data from your MSM server and uploads it to a cloud platform for monitoring, analytics, and reporting.

Do I need to be a programmer to use it?

No. The Data Collector has a user-friendly desktop interface that requires no programming knowledge. If you can use basic desktop applications, you can use this tool.

Is my data secure?

Yes. All data transfers use encrypted SSH and HTTPS connections. Your configuration file contains sensitive credentials and should be protected accordingly.

Installation and Setup

What do I need to install?

You need Python 3.11 or higher and the application files. All dependencies are installed automatically. See the Installation Guide for details.

Where do I get my configuration details?

Contact your system administrator for MSM server connection details, database credentials, and cloud platform URL.

Can I run this on any computer?

Yes. The Data Collector works on Windows, macOS, and Linux. You need Python 3.11+ and network access to your MSM server.

How much disk space do I need?

For the application: ~100 MB For offline extractions: ~1 MB per 10,000 records extracted

Can multiple people use the same installation?

Each user should have their own installation with their own configuration file for security and audit purposes.

Usage Questions

Which mode should I use?

How long does an extraction take?

It depends on: - Amount of data - Network speed - Server performance - Mode used (online vs offline)

Typical: 5,000-10,000 records per minute in Online Mode

Can I run extractions in the background?

The application must stay open during extraction, but you can minimize the application window. Don't close the window or terminal completely.

Can I stop an extraction and resume later?

Yes. Use the Stop button, note the last processed timestamp from logs, and start a new extraction from that point.

How often should I run extractions?

This depends on your data collection needs. Common schedules: - Daily for regular monitoring - Weekly for less critical data - On-demand for specific analyses

Can I extract data from multiple MSM servers?

You'll need separate configuration files for each server. Stop the current extraction before switching to a different server configuration.

Data Questions

What data is extracted?

The Data Collector extracts telemetry data from your MSM database's telemetry table. The specific data depends on your MSM configuration.

How far back can I extract data?

You can extract data from any date in your MSM database. Older data may take longer to extract depending on database size.

Will extraction affect my MSM server performance?

Extraction uses minimal resources. The application reads data through SSH tunnel with configurable batch sizes to minimize impact.

Can I extract data while MSM is recording?

Yes. The extraction doesn't interfere with active data recording on your MSM server.

What if I accidentally extract the same data twice?

The system doesn't automatically detect duplicates. Always track your last extraction date to avoid re-extracting data.

How do I know what date range to extract?

  • For first extraction: Start from your earliest needed date
  • For subsequent extractions: Use the end date from your last extraction as the start date
  • Check extraction logs for last processed timestamps

Technical Questions

What database does MSM use?

MSM uses PostgreSQL. The Data Collector connects via SSH tunnel to ensure secure access.

What format is the extracted data?

Data is extracted in CSV format and compressed with gzip (.csv.gz files) for efficient storage and transfer.

Where are files stored?

  • Offline extractions: offline_extractions/ folder
  • Logs: logs/ folder
  • Configuration: config.yaml file in application root

Can I customize the extraction?

Basic customization (date ranges, batch sizes) is available in the interface. Advanced customization requires editing the configuration file.

What happens if my computer crashes during extraction?

In Online Mode: Note the last successful batch from logs and restart from that point. In Offline Mode: Completed files are safe. Restart extraction for remaining data.

Does this work with VPN?

Yes, as long as your VPN allows access to the MSM server and cloud platform.

Troubleshooting

Why is extraction so slow?

Common causes: - Large date range - Slow network connection - High server load - Large batch sizes

Try reducing batch size or using Offline Mode.

Why do I get authentication errors?

  • Check username and password in config.yaml
  • Verify account is active
  • Ensure no typos or extra spaces in credentials

What if uploads keep failing?

  1. Verify internet connection
  2. Check cloud platform URL in config
  3. Try Offline Mode then upload later
  4. Contact administrator about cloud platform status

Where are error logs?

Check logs/extraction.log for detailed error information.

Can't find offline extraction files?

Check the offline_extractions/ folder in your application directory. Files are named extraction_YYYYMMDD_HHMMSS_part_NNN.csv.gz.

Best Practices

Should I delete old extraction files?

Yes, but only after confirming successful upload to the cloud platform. Keep files until you verify data is accessible in the cloud.

How do I avoid extracting duplicate data?

Always use the end timestamp of your previous extraction as the start timestamp for your next extraction.

What's the best batch size?

  • Fast connection (50+ Mbps): 2000-5000
  • Medium connection (10-50 Mbps): 1000-2000
  • Slow connection (<10 Mbps): 500-1000

Should I use Online or Offline Mode?

  • Online Mode: Best for small-to-medium datasets with stable internet
  • Offline Mode: Best for large datasets or unreliable connections

How do I backup my configuration?

Copy your config.yaml file to a secure location. Keep it updated when credentials change.

Security Questions

Is my password visible in config.yaml?

Yes, credentials are stored in plain text in config.yaml. Protect this file with appropriate file permissions and don't share it.

Can others see my data?

Only if they have access to your computer and the application folder. Use appropriate file permissions to protect sensitive data.

Should I share my config.yaml file?

No. Never share your configuration file as it contains passwords and sensitive connection details.

What happens to my data after upload?

Data is stored securely in the cloud platform where authorized users can access it for monitoring and analytics. Contact your administrator about data retention policies.

Updates and Maintenance

How do I update to a new version?

Your administrator will provide new version files. Back up your config.yaml, replace application files, and restore your configuration.

Do I need to update Python?

Only if a new version requires it. Check release notes from your administrator.

How do I know if there's a new version?

Your administrator will notify you of important updates. There's no automatic update check in the application.

Still Have Questions?

If your question isn't answered here:

  1. Check the Troubleshooting guide
  2. Review the specific mode documentation:
  3. Online Mode
  4. Offline Mode
  5. Upload Mode
  6. Contact your system administrator for technical support