The VFS transport implementation is based on Apache Commons VFS implementation. VFS (Virtual File System) transport implementation is a module which belongs to the Apache Synapse project. It has a set of service level parameters that needs to be specified for each service. VFS service level parameters and their descriptions can be found in [1] and the endpoint formats can be found in [2].

Following is a SFTP sample for ESB 3.0.1. It copies a file from one SFTP location to another SFTP folder. SFTPVFSProxy is the proxy service which copies file from one SFTP location to another.

Applies to:

Create Directories in the SFTP server

Create the directories named "in", "out" and "original" in the SFTP Server.

NOTE: Remember to update the proxy service configuration with the paths of these directories.

Registering Transport Listener and Transport Sender

1). Uncomment the VFS Transport Listener from the axis2.xml (which resides in ESB_HOME/repository/conf/ directory)

 <transportReceiver name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportListener"/>

2). Uncomment the VFS Transport Sender from the axis2.xml (which resides in ESB_HOME/repository/conf/ directory)

 <transportSender name="vfs" class="org.apache.synapse.transport.vfs.VFSTransportSender"/>

Proxy Configuration

1). Go to the ESB_HOME/bin and run the script which starts wso2server.
Eg. wso2server.bat for windows environments
      wso2server.sh for linux environments

2). The server will then start. You can access the management console using the following URL [8].

3). Log into the Mangement Console using following credentials.
      username: admin
      password: admin

4). Select the Source View from the ESB Management Console and add the following proxy configuration to the Synapse configuration. 

NOTE: Remember to update the directory locations according to your SFTP server.

<proxy name="SFTPVFSProxy" transports="vfs" startOnLoad="true" xmlns="http://ws.apache.org/ns/synapse">
      <target>
          <inSequence>
              <log level="full"/>
              <property name="File" expression="fn:concat('test-', get-property('transport', 'FILE_PATH'))" scope="default"/>
              <property name="transport.vfs.ReplyFileName" expression="fn:concat(fn:substring-after(get-property('MessageID'), 'urn:uuid:'), '.xml')" scope="transport"/>
              <property name="OUT_ONLY" value="true"/>
              <send>
                  <endpoint name="endpoint_urn_uuid_A1546EFFD75FC9CCED785986339425964585275">
                      <address uri="vfs:sftp://heshan:[email protected]/home/heshan/out"/>
                  </endpoint>
              </send>
          </inSequence>
      </target>
      <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
      <parameter name="transport.PollInterval">15</parameter>
      <parameter name="transport.vfs.MoveAfterProcess">vfs:sftp://heshan:[email protected]/home/heshan/original</parameter>
      <parameter name="transport.vfs.FileURI">vfs:sftp://heshan:[email protected]/home/heshan/in/test.xml</parameter>
      <parameter name="transport.vfs.MoveAfterFailure">vfs:sftp://heshan:[email protected]/home/heshan/original</parameter>
      <parameter name="transport.vfs.FileNamePattern">.*.xml</parameter>
      <parameter name="transport.vfs.ContentType">application/xml</parameter>
      <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>

Running the Sample

Copy the following XML (test.xml) file to the directory named "in" which is in your SFTP server. Then it will be moved to the directory named "out".

Following is the XML file (test.xml) that is being moved.

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="https://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="https://www.w3.org/2005/08/addressing">
   <soapenv:Body>
       <getQuote xmlns="https://services.samples">
           <request>
               <symbol>IBM</symbol>
           </request>
       </getQuote>
   </soapenv:Body>
</soapenv:Envelope>

Troubleshooting

Resolving UnknownHostKey Exception

Earlier, we looked at how to do a SFTP file transfer with WSO2 ESB's Virtual File System (VFS) transport. When running the above sample, for example when we specified the VFS endpoint by giving the host name (instead of the IP address). Then, you might come across the following exception (UnknownHostKey).

Exception thrown in the command prompt;

[2010-01-19 22:03:58,336] ERROR - VFSTransportListener cannot resolve fileObject
org.apache.commons.vfs.FileSystemException: Could not connect to SFTP server at "sftp://user1:***@ftps.example.com/".
        at org.apache.commons.vfs.provider.sftp.SftpFileProvider.doCreateFileSystem(SftpFileProvider.java:106)
        at org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.getFileSystem(AbstractOriginatingFileProvider.java:103)
        at org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:82)
        at org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:66)
        at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:692)
        at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:648)
        at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:604)
        at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportListener.java:179)
        at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:153)
        at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:104)
        at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingTransportListener.java:76)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:58)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)Caused by: org.apache.commons.vfs.FileSystemException: Could not connect to SFTP server at "ftps.example.com".
        at org.apache.commons.vfs.provider.sftp.SftpClientFactory.createConnection(SftpClientFactory.java:233)
        at org.apache.commons.vfs.provider.sftp.SftpFileProvider.doCreateFileSystem(SftpFileProvider.java:95)
        ... 14 more
Caused by: com.jcraft.jsch.JSchException: UnknownHostKey: ftps.example.com. DSA key fingerprint is 62:fb:a5:c7:1a:34:f4:05:7a:e8:06:b9:57:e5:de:e4
        at com.jcraft.jsch.Session.checkHost(Session.java:706)
        at com.jcraft.jsch.Session.connect(Session.java:307)
        at com.jcraft.jsch.Session.connect(Session.java:150)
        at org.apache.commons.vfs.provider.sftp.SftpClientFactory.createConnection(SftpClientFactory.java:229)
        ... 15 more

Explanation

Apache Commons-VFS [2] uses JSCH [5] for its underlying SSH layer. JSCH uses the location $HOME/.ssh for the known_hosts file by default.
Eg. location is $HOMEPATH/.ssh

Therefore, since the host that you are trying to connect with is not in the known_hosts file, Synapse has no idea whether the SFTP hosts can be trusted. Therefore, we need to add the host to the known_hosts file.

Workaround

1) Open SSH using the above location too by default.
2) If you open an SSH session in the target host, it will ask to add the credentials to this file. However, the session will fail as you should not have permission on an SFTP server to do this; but that is alright.
3) After the above modification, you now have the updated known_hosts file.

Now the SFTP sample will work with this host.

Configure your SFTP server with write_enable

The VFS transport uses a file locking mechanism to make sure that a single file is being processed by a single listener in a given situation. Therefore, the VFS transport will add a lock file and when the processing completes, it will remove the lock file. Therefore, it is mandatory to have the above mentioned property enabled in your SFTP server.

Eg. If the SFTP server that you are using is vsftpd [6], then set the write_enable property to "YES".

References
If you are interested in trying out more VFS samples, refer [3] and [4].

[1] - https://wso2.org/project/esb/java/3.0.1/docs/transports/transports-catalog.html#VfsTrp
[2] - https://commons.apache.org/vfs/filesystems.html#SFTP
[3] - https://wso2.org/project/esb/java/3.0.1/docs/samples/transport_samples.html#Sample254
[4] - https://wso2.org/project/esb/java/3.0.1/docs/samples/transport_samples.html#Sample255
[5] - https://www.jcraft.com/jsch/
[6] - https://vsftpd.beasts.org/
[7] - https://wso2.org/downloads/esb
[8] - https://10.100.1.164:9443/carbon/