The Security Stakes for Financial Data Pipelines
A compliance officer at a regional broker-dealer discovered the problem during a routine internal audit. A former operations analyst — who had left the firm eight months earlier — still had active SFTP credentials for three custodian data feeds. His access had never been revoked. The credentials had been used twice in the intervening months, from an IP address the firm did not recognize. Neither access event had triggered an alert because nobody was monitoring SFTP access logs at all.
The firm could not determine whether client position data had been exfiltrated.
That scenario is not unusual. Financial data pipelines are often the least-secured part of a firm's technology infrastructure. The reason is historical: financial data sharing evolved through informal practices — emailing spreadsheets, FTP sites, manual file drops — that predate modern security standards. Many firms have layered security controls onto fundamentally insecure practices without addressing the underlying architecture.
This article covers the essential security practices for financial data pipelines and the specific gaps where institutional firms remain most exposed.
File Transfer Protocol Security: SFTP, FTPS, and API Compared
FTP — Never Acceptable
Plain FTP transmits data and credentials in cleartext. Anyone with network access can intercept the data stream and capture both the content of transferred files and the credentials used to authenticate.
There is no scenario in which plain FTP is acceptable for financial data. It violates SOC 2, PCI DSS, and most institutional security policies. If your firm still uses plain FTP for any financial data transfer, it is a priority remediation item — not a future roadmap item.
SFTP (SSH File Transfer Protocol)
SFTP uses SSH (Secure Shell) to encrypt both authentication and data transfer. It is the minimum acceptable standard for financial data file transfer.
Key security practices for SFTP:
- Use SSH key-based authentication rather than password authentication. Passwords can be brute-forced or leaked; SSH keys are significantly more resistant to both attacks.
- Rotate SSH keys on a defined schedule — annually at minimum, quarterly for high-sensitivity connections.
- Restrict SFTP server access by IP allowlist where possible. Connections from unexpected IPs should be blocked, not just logged.
- Monitor SFTP access logs for anomalous patterns: off-hours access, unusual file sizes, unexpected connection sources. Logging that nobody reads is not a control.
FTPS (FTP over SSL/TLS)
FTPS adds TLS encryption to standard FTP. It is more secure than plain FTP but has operational complexity disadvantages compared to SFTP: FTPS requires multiple ports to be open on firewalls (control port plus data ports), which complicates firewall management and increases attack surface. For new implementations, SFTP is generally preferred over FTPS.
API-Based Transfer
Modern REST APIs using HTTPS (TLS 1.2 or 1.3) provide the strongest security foundation for financial data transfer. API authentication using OAuth 2.0 or API keys with proper rotation, combined with HTTPS transport encryption, provides better security than file-based transfer methods.
APIs also enable finer-grained access control: a file transfer gives access to an entire file, while an API can return only the data a specific caller is authorized to see. That difference matters when you are thinking about least-privilege access.
Encryption Standards
Encryption in Transit
All financial data in transit must be encrypted. Current standards:
- TLS 1.2 or 1.3 for HTTPS and FTPS connections. TLS 1.0 and 1.1 are deprecated and must be disabled — not just discouraged.
- SSH with AES-256 for SFTP connections.
- Perfect Forward Secrecy (PFS) cipher suites should be preferred. These ensure that a compromised private key does not allow decryption of previously captured traffic.
Encryption at Rest
Financial data stored at rest must be encrypted:
- AES-256 is the current standard for data at rest.
- Encryption keys must be managed separately from encrypted data. A data store with embedded encryption keys provides little protection against a determined attacker who gains access to the storage layer.
- Key management systems (AWS KMS, Azure Key Vault, HashiCorp Vault) should be used for key storage and rotation — not config files, not spreadsheets.
- Database-level encryption protects against physical media theft. Application-level encryption provides additional protection against database access by unauthorized internal users.
Access Control Patterns
Principle of Least Privilege
Every user, service account, and system integration should have access only to the data it needs — and nothing more.
In practice, this means:
- Custodian SFTP connections use accounts with access only to their specific data directories, not the full file system.
- Operations staff can see data relevant to their function but cannot access client data outside their purview.
- API keys are scoped to specific operations (read-only vs. read-write, specific data types).
This is not a one-time setup. As roles change and integrations are added, access scope creep is common. Budget time for quarterly access reviews.
Role-Based Access Control (RBAC)
Institutional financial firms should implement role-based access control for financial data:
- Define roles that correspond to job functions — portfolio manager, operations analyst, compliance officer, auditor.
- Assign permissions to roles, not to individuals. Individual-level permission assignment does not scale and is consistently under-maintained.
- Review role assignments quarterly. Revoke access promptly when staff change roles or leave the firm. "Promptly" means same day, not end of quarter.
Multi-Factor Authentication
Any human access to financial data systems should require multi-factor authentication. This includes:
- Internal data portals and dashboards
- Cloud storage consoles
- Administrative access to data pipeline systems
- Vendor portals that provide access to client or position data
MFA alone does not prevent all breaches. But it significantly raises the cost of credential-based attacks, which remain the most common vector.
Audit Logging Requirements
A complete audit trail for financial data operations must capture:
- Who accessed data (user, service account, or API key)
- What data was accessed (file, table, query, or API endpoint)
- When the access occurred (timestamp with timezone)
- From where the access originated (IP address, device)
- What action was performed (read, write, delete, export)
Audit logs must be:
- Tamper-evident: logs stored in systems where they can be modified are not reliable audit trails. Append-only logging with cryptographic integrity checks is the right approach.
- Retained for the required period. SEC Rule 17a-4 requires 6 years for most records, with the first 2 years in immediately accessible form.
- Reviewable: logs must be searchable and filterable to support incident investigation and regulatory examination. A log you cannot search is a log you cannot use.
Before You Sign Off on Your Pipeline Security
Here is the question to ask before you consider your financial data pipeline secure: if a former employee's credentials were used to access your SFTP server tonight, would your team know by 9 AM tomorrow?
If not, your access monitoring gap is more urgent than any other item on your security roadmap.
Common Security Gaps in Financial Data Pipelines
Credential Management
The most common security gap is poor credential management: SFTP passwords and API keys stored in configuration files, spreadsheets, or shared email accounts. This is where the broker-dealer example at the top of this article went wrong.
Credentials should be stored in a secrets management system, rotated on a defined schedule, and audited for access. Any credential that has not been rotated in over 12 months should be treated as potentially compromised until proven otherwise.
Stale Access
Former employees whose access was not revoked, third-party vendors whose contracts ended but whose credentials remain active, service accounts for decommissioned systems — all represent unnecessary access that expands your attack surface without providing any operational value.
Conduct a full access audit. You will find accounts you did not know existed.
Unencrypted Email Delivery
Financial data sent via standard email is transmitted without guaranteed encryption. If email delivery of financial data is necessary at all, it should use encrypted email (S/MIME or PGP) or be replaced with secure file transfer. Many fund administrators still deliver statements via email attachment. This is a risk you should flag and work to migrate away from.
Missing Data-in-Transit Monitoring
Firms that monitor network perimeters but do not monitor data-in-transit content cannot detect exfiltration events where data is leaving the organization in encrypted channels. Perimeter monitoring tells you who is connecting. It does not tell you what they are taking.
The Hard Truth About Financial Data Security
| What teams assume | What actually happens |
|---|---|
| SFTP means our file transfers are secure | SFTP encrypts the transfer but says nothing about who has credentials, whether those credentials have been rotated, or whether access is being monitored |
| Our cloud infrastructure provider handles security | Cloud providers secure the infrastructure layer; access controls, credential management, and audit logging within your applications are your responsibility |
| We would notice a data breach quickly | The average time to detect a breach in financial services is over 200 days; most are discovered by third parties, not internal monitoring |
| Audit logs are for compliance, not operations | Audit logs are your primary forensic resource after an incident; teams that treat them as compliance checkbox items consistently lack the data they need during investigations |
| Security controls slow down operations | The operational cost of proper credential management and access review is measured in hours per quarter; the operational cost of a breach is measured in months of remediation |
FAQ
Is SFTP sufficient for financial data transfers, or do we need something more?
SFTP is sufficient as a transport mechanism when it is properly configured — SSH key-based authentication, IP allowlisting, and active access log monitoring. The issue is usually not the protocol but the configuration and monitoring around it. An SFTP connection with password authentication and no access monitoring is not secure, regardless of the protocol.
How often should we rotate credentials for data source connections?
Annually at minimum for standard connections, quarterly for high-sensitivity or externally-facing connections. Any credential associated with a vendor relationship that has ended should be revoked immediately, not at the next scheduled rotation. Automate rotation wherever possible — manual rotation processes are consistently skipped or delayed.
What does SOC 2 Type II certification actually mean for a data platform?
SOC 2 Type II means an independent auditor reviewed the platform's security controls over a sustained period — typically six months to a year — and verified they operated as designed. This is meaningfully stronger than SOC 2 Type I, which is a point-in-time snapshot. For a financial data platform, SOC 2 Type II is the minimum certification to require, not a bonus feature.
Do we need application-level encryption in addition to database-level encryption?
Yes, if you have any concern about unauthorized internal access to the database layer. Database-level encryption protects against physical media theft and external attackers; it does not protect against a database administrator or a compromised database account reading data directly. Application-level encryption provides an additional layer of protection for your most sensitive data elements — account numbers, position details, client identifiers.
How do we handle security for fund administrators that still deliver data via email?
Flag email delivery as a risk in your vendor management process and push administrators toward SFTP or portal delivery. In the interim, use encrypted email (S/MIME or PGP) if the administrator supports it, or establish a secure portal where they can upload files. Do not accept unencrypted email delivery of position or client data as a long-term practice.
What is the minimum audit log retention period for a registered investment advisor?
Six years under SEC Rule 17a-4, with the first two years in an immediately accessible format. This applies to books and records broadly, including data access logs. Consult your compliance counsel for requirements specific to your registration type and regulatory obligations — state-registered advisors may have different requirements.
Building a secure financial data pipeline is not a one-time project. It is an ongoing operational discipline. The firms that maintain the strongest security posture treat credential management, access review, and audit log monitoring as regular operational responsibilities — not annual compliance checkboxes.