Transferring local directory structures to remote servers securely is a common requirement for deployment scripts and backup utilities. While Python does not include built-in SCP (Secure Copy Protocol) capabilities, developers can implement this functionality using the Paramiko library to interact with the SFTP subsystem or by wrapping the native system scp command.

Implementing Recursive SFTP Uploads with Paramiko

Paramiko provides a comprehensive SSHv2 implementation. To replicate the recursive behavior of scp -r, one must establish an SFTP client, traverse the local directory tree, and mirror the directory structure on the remote server before file transfer.

import os
import paramiko
import sys

def recursive_upload(local_path, remote_path, hostname, username, password, port=22):
    """
    Establishes an SFTP connection and uploads a directory recursively.
    """
    # Initialize transport and client
    transport = paramiko.Transport((hostname, port))
    try:
        transport.connect(username=username, password=password)
        sftp = paramiko.SFTPClient.from_transport(transport)

        def _traverse_and_upload(current_local, current_remote):
            for item in os.listdir(current_local):
                local_item = os.path.join(current_local, item)
                remote_item = os.path.join(current_remote, item)
                
                if os.path.isfile(local_item):
                    print(f"Uploading {local_item} -> {remote_item}")
                    sftp.put(local_item, remote_item)
                elif os.path.isdir(local_item):
                    try:
                        # Check if directory exists to avoid errors
                        sftp.stat(remote_item)
                    except IOError:
                        print(f"Creating directory {remote_item}")
                        sftp.mkdir(remote_item)
                    # Recurse into subdirectory
                    _traverse_and_upload(local_item, remote_item)

        # Ensure the base remote directory exists
        try:
            sftp.stat(remote_path)
        except IOError:
            sftp.mkdir(remote_path)
            
        _traverse_and_upload(local_path, remote_path)

    finally:
        transport.close()

# Configuration parameters
source_dir = './local_project'
target_dir = '/var/www/html'
server_ip = '192.168.1.10'
user = 'admin'
secret = 'secure_password'

# Execute transfer
recursive_upload(source_dir, target_dir, server_ip, user, secret)

Invoking Native SCP via Subprocess

An alternative approach that often yields better performance is to delegate the transfer to the system's native scp utility using the subprocess module. This method leverages the -r flag to handle directory recursion automatically, eliminating the need for manual directory traversal logic in Python.

import subprocess
import shutil

def transfer_using_system_scp(src_dir, dest_host, dest_user, dest_path):
    """
    Uses the system's scp command to copy a directory recursively.
    Requires 'scp' to be available in the system PATH.
    """
    # Verify scp is available
    if not shutil.which('scp'):
        raise EnvironmentError("The 'scp' command is not installed or not found in PATH.")

    # Construct the destination string: user@host:path
    destination = f"{dest_user}@{dest_host}:{dest_path}"
    
    # Build the command list
    command = ['scp', '-r', src_dir, destination]

    print(f"Executing: {' '.join(command)}")
    
    try:
        # Run the command and check for errors
        process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = process.communicate()
        
        if process.returncode != 0:
            error_msg = stderr.decode('utf-8').strip()
            raise Exception(f"SCP transfer failed: {error_msg}")
            
        print("Transfer completed successfully.")
        
    except KeyboardInterrupt:
        process.kill()
        raise
    except Exception as e:
        raise e

# Example Usage
local_folder = './assets'
remote_server = 'example.com'
remote_user = 'deploy_user'
remote_directory = '/opt/assets'

transfer_using_system_scp(local_folder, remote_server, remote_user, remote_directory)

Security Considerations

Hardcoding credentials in scripts poses a significant security risk. For production environments, it is advisable to utilize SSH key-based authentication. When using subprocess, ensure that the command arguments are sanitized if they are derived from external input to prevent command injection vulnerabilities.