File Storage Connection Options

If you are querying a FileSource and additional connection information needs to be provided to access the file storage system, include the options property in the query and define the necessary storage-specific connection parameters. The parameters that the GDI supports for each type of storage system are pulled directly from the Java API for that system. The supported properties for each storage type are listed below.

Amazon S3

s:options [ 
   s:accessKey "string" ;
   s:region "string" ;
   s:secretKey "string" ;
   s:serviceName "string" ;
   s:sessionToken "string" ;
   s:createBucket boolean ;
   s:disableChunkedEncoding boolean ;
   s:serverSideEncryption boolean ;
   s:useHttps boolean ;
] ;
Option Data Type Description
accessKey string The accessKey property can be included to specify the access key.
region string The region property can be included to specify the region.
secretKey string The secretKey property can be included to specify the secret key.
serviceName string For connections to AWS service endpoints, the serviceName property can be included to specify the service name.
sessionToken string The sessionToken property can be included to specify the session token.
createBucket boolean Refer to the S3 API documentation.
disableChunkedEncoding boolean For increased performance, Amazon S3 requests use chunked encoding by default. To disable chunked encoding, you can include s:disableChunkedEncoding true in the query.
serverSideEncryption boolean Refer to the S3 API documentation.
useHttps boolean Refer to the S3 API documentation.

FTP & FTPS

s:options [ 
   s:autodetectUtf8 boolean ;
   s:connectTimeout int ;
   s:controlEncoding "string" ;
   s:dataTimeout int ;
   s:defaultDateFormat "string" ;
   s:entryParser "string" ;
   s:fileType "string" ;
   s:passiveMode boolean ;
   s:proxy "string" ;
   s:recentDateFormat "string" ;
   s:remoteVerification boolean ;
   s:serverLanguageCode "string" ;
   s:serverTimeZoneId "string" ;
   s:shortMonthNames "string" ;
   s:socketTimeout int ;
   s:userDirIsRoot boolean ;
   s:dataChannelProtectionLevel "string" ;
   s:ftpsMode "string" ;
   s:keyManager "string" ;
   s:trustManager "string" ;
] ;
Option Data Type Description
autodetectUtf8 boolean For FTP connections, the autodetectUtf8 property can be included to indicate whether the FTP server is set to UTF-8 mode or Auto-detect encoding.
connectTimeout int For FTP connections, you can include the connectTimeout property to specify the maximum number of seconds to hold a connection before timing out.
controlEncoding string Refer to the FTP API documentation.
dataTimeout int For FTP connections, you can include the dataTimeout property to specify the maximum number of seconds to transfer data before timing out.
defaultDateFormat string Refer to the FTP API documentation.
entryParser string Refer to the FTP API documentation.
fileType string Refer to the FTP API documentation.
passiveMode boolean For FTP connections, the passiveMode property can be included to indicate whether the data transfer mode is passive or active. If you use passive mode, set passiveMode to true (s:passiveMode true).
proxy string If you are using an FTP proxy, include the proxy property to specify the proxy connection details.
recentDateFormat string Refer to the FTP API documentation.
remoteVerification boolean For FTP connections, the remoteVerification property can be included to indicate whether remote authentication is enabled. If you use remote authentication, set remoteVerification to true (s:remoteVerification true).
serverLanguageCode string If the FTP server language is not set to English, include the serverLanguageCode property to specify the language code for the server. For example, s:serverLanguageCode "ES".
serverTimeZoneId string For FTP connections, the serverTimeZoneId property can be included to specify the timezone ID for the server.
shortMonthNames string Refer to the FTP API documentation.
socketTimeout int For FTP connections, you can include the socketTimeout property to specify the maximum number of seconds to transfer data before timing out.
userDirIsRoot boolean Refer to the FTP API documentation.
dataChannelProtectionLevel string For FTPS connections, the dataChannelProtectionLevel property specifies the Data Channel Protection Level for the server.
ftpsMode string For FTPS connections, the ftpsMode property specifies whether the FTPS is in implicit or explicit mode.
keyManager string For FTPS connections, the keyManager property specifies the KeyManager value for making an SSL connection to the server.
trustManager string For FTPS connections, the trustManager property specifies the TrustManager value for the SSL connection to the server.

Google Cloud Storage

s:options [ 
   s:serviceAccountKey "string" ;
] ;
Option Data Type Description
serviceAccountKey string For connections to GCS, the serviceAccountKey property can be included to specify the key for the service account.

HDFS

s:options [ 
   s:configName "string" ;
   s:configPath "string" ;
   s:configURL "string" ;
] ;
Option Data Type Description
configName string For connections to HDFS, the configName property can be included to specify the name of the configuration file to read.
configPath string For connections to HDFS, the configPath property can be included to list the path to the specified configuration file.
configURL string Refer to the HDFS API documentation.

SFTP

s:options [ 
   s:compression "string" ;
   s:configRepository "string" ;
   s:fileNameEncoding "string" ;
   s:identityProvider "string" ;
   s:identityRepositoryFactory "string" ;
   s:keyExchangeAlgorithm "string" ;
   s:knownHosts "string" ;
   s:loadOpenSSHConfig boolean ;
   s:preferredAuthentications "string" ;
   s:sessionTimeout int ;
   s:strictHostKeyChecking "string" ;
   s:userInfo "string" ;
] ;
Option Data Type Description
compression string Refer to the SFTP API documentation.
configRepository string Refer to the SFTP API documentation.
fileNameEncoding string Refer to the SFTP API documentation.
identityProvider string Refer to the SFTP API documentation.
identityRepositoryFactory string Refer to the SFTP API documentation.
keyExchangeAlgorithm string For SFTP connections, you can include the keyExchangeAlgorithm property to specify the key exchange algorithm to use.
knownHosts string Refer to the SFTP API documentation.
loadOpenSSHConfig boolean For SFTP connections, the loadOpenSSHConfig property indicates whether to read the ~/.ssh/config file.
preferredAuthentications string For SFTP connections, the preferredAuthentications property can be included to specify the authentication order to use.
sessionTimeout int For SFTP connections, you can include the sessionTimeout property to specify the maximum number of seconds to leave the session open before timing out.
strictHostKeyChecking string For SFTP connections, you can include the strictHostKeyChecking property to specify how host keys are checked.
userInfo string Refer to the SFTP API documentation.

WebDAV

s:options [ 
   s:creatorName "string" ;
   s:versioning boolean ;
] ;
Option Data Type Description
creatorName string For WebDAV connections, the creatorName property can be included to add a description of the creator of the resource.
versioning boolean Refer to the WebDAV API documentation.

Related Topics