SFTP Connector

Connection Method

Used for OneTrust Data Discovery, the secure File Transfer Protocol (SFTP) Connector helps users who leverage the protocol for file transport on their servers to scan, retrieve, and classify data for discovery business needs.

Note:

This connector is currently under Public Preview. Contact OneTrust Support to request early access.

Pod Residency

storage-scanner

Connection Method

sFTP protocol with Basic Authentication or Public Key Authentication.

Supported Scan Actions

Scan Type

Description

Catalog

This scan type will fetch the list of folders and files on the server. It collects the metadata also for these objects and sends back to the main OneTrust application.

Catalog & Scan

This fetches the list of files and after that this can extract every file. Metadata is collected and then content extraction. It will return personally identifiable information (PII), data elements (DE), and terms as per the classification label identified.

Credential and System Permissions

OneTrust recommends to adhere to your organization's security and infrastructure policy and include system engineer resources or other qualified subject matter experts to set up credentials on data sources. For specific instructions for how to enable these credentials, refer to your data source's implementation documentation, knowledge base and/or support resources.

This connector relies on Basic Authentication with username and password credentials or Private Key Authentication. Use the Build Credential JSON option to create either one of these credential offerings.

Basic Authentication

JSON

JSON 
#Default port is 22

{
  "host": "<Hostname/IP>",
  "port": "<port>",
  "username": "<username>",
  "password": "<password>"
}
Hostname

JSON 
{
  "host": "hostname.sftp.example.com",
  "port": "22",
  "username": "user",
  "password": "Password"
}
IP Address

JSON 
{
  "host": "192.168.1.100",
  "port": "22",
  "username": "user",
  "password": "Password"
}
Azure Files SFTP

The generated connection string will look similar to the following, where in this case the username is the string before the "@" with the host following.

JSON 
sftpserver.user@sftpserver.blob.core.windows.net

{
  "host": "sftpserver.blob.core.windows.net",
  "port": "22",
  "username": "sftpserver.user",
  "password": "Password"
}

Private Key Authentication

JSON with Passphrase

JSON 
#Format
#Default port is 22
{
  "host": "<Hostname/IP>",
  "port": "<port>",
  "username": "<username>",
  "privatekey": "<private_key>",
  "passphrase": "<passphrase>"
}

#Example
{
  "host": "sftpserver.example.com",
  "port": "22",
  "username": "username",
  "privatekey": "-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: AES-128-CBC,AA349E4F1 <rest_of_private_key> 3aF8bIJ/1F\n-----END RSA PRIVATE KEY-----",
  "passphrase": "securePassphrase"
}
JSON without Passphrase

JSON 
#Format
#Default port is 22
{
  "host": "<Hostname/IP>",
  "port": "<port>",
  "username": "<username>",
  "privatekey": "<private_key>"
}

#Example
{
  "host": "sftpserver.example.com",
  "port": "22",
  "username": "username",
  "privatekey": "-----BEGIN RSA PRIVATE KEY-----\nMIIG5AIBAAKCAYEA0gC <rest_of_private_key>  Q43ljt8FH7BCRUrw67Q==\n-----END RSA PRIVATE KEY-----"
}

Scan Profile Support

Note:

For additional details on scan profile configuration and management, see Setting Up Scan Profiles.

Limits

  • Max number of folders

  • Max number of files per folder

  • Maximum Depth: once in folder, A, and B, C folders, define a depth 2, only goes two folders down.

  • Minimum file size

  • Maximum file size

  • Include/Exclude File Types

  • Include/Exclude File Paths

Collected Objects

Objects

  • Folder: Nested folders in share

  • Files: Files directly inside a share or folders

    Note:

    Hidden files are supported and scanned.

Collected Fields

  • Name

  • Modified Date

  • File Owner

  • Size on Disk (Files Only)

  • Content Type

  • Parent Object

  • Path

  • Permission Set

Limitations of Incremental Scans

When a file is moved to an SFTP server, the file's original creation and modification dates are typically retained. Consequently, the file stats will reflect the original creation date, not the date the file was transferred to the server. Incremental scans that rely on the file's creation or modification date will be affected by this behavior. This impacts connectors for SFTP, SMB, and Slack data sources.

Incremental scan is designed to detect new or updated files based on these dates. It may fail to recognize the actual date of transfer to the SFTP server. As a result, files moved at a later time might not be detected as new or updated by the incremental scan.

Troubleshooting

Timeout

Indicates possible incorrect authentication. Check your username and password, plus examine if you have the right port number.

CODE 
ERROR --- [  sftpjob1-P-1] c.o.d.s.s.c.s.p.SFTPJobEventProcessor  : JobId: 6, 
JobClass: SFTP: Failed processing. Error: Code:[CLIENT_ERROR] Source:[Connection Error] 
Message:[ConnectException: Connection timed out] Arguments:[[]] Date:[2024-06-03T10:05:26]
java.util.concurrent.CompletionException: com.onetrust.discovery.scanner.common.exceptions.DataDiscoveryException: 
ConnectException: Connection timed out
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.onetrust.discovery.scanner.common.exceptions.DataDiscoveryException: ConnectException: Connection timed out

Unknown Host

The host in the credential JSON is incorrect. Check that you are setting only the host, without the “username@” prefix, and that it is the correct host or IP address.

CODE 
storage-scanner: Job failed. Code:[CLIENT_ERROR] Source:[Connection Error] 
Message:[UnknownHostException: username0@hostname.example.com]  
Date:[2024-06-04T03:00:21]

Incorrect Username Format

Check the value for username in the credential JSON. It should only have the username, without the host you are trying to connect to.

CODE 
storage-scanner: Job failed. Code:[CLIENT_ERROR] Source:[Connection Error]
Message:[JSchException: SSH_MSG_DISCONNECT: 15 The SSH username format is invalid. - RequestId: 
Time:2024-06-04T02:58:10.2672122Z ]  Date:[2024-06-04T02:58:10]