summaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
-rw-r--r--.gitignore2
-rw-r--r--.travis.yml2
-rw-r--r--Makefile2
-rw-r--r--NEWS62
-rw-r--r--README20
-rw-r--r--fabfile.py13
-rw-r--r--paramiko/__init__.py13
-rw-r--r--paramiko/client.py47
-rw-r--r--paramiko/config.py61
-rw-r--r--paramiko/file.py4
-rw-r--r--paramiko/message.py3
-rw-r--r--paramiko/packet.py16
-rw-r--r--paramiko/proxy.py91
-rw-r--r--paramiko/sftp_client.py130
-rw-r--r--paramiko/sftp_file.py9
-rw-r--r--paramiko/ssh_exception.py17
-rw-r--r--paramiko/transport.py8
-rw-r--r--requirements.txt2
-rw-r--r--setup.py2
-rw-r--r--tests/test_buffered_pipe.py8
-rwxr-xr-xtests/test_sftp.py13
-rw-r--r--tests/test_transport.py7
-rw-r--r--tests/test_util.py63
-rw-r--r--tests/util.py10
-rw-r--r--tox.ini6
25 files changed, 486 insertions, 125 deletions
diff --git a/.gitignore b/.gitignore
index 3283ff38..4b578950 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,7 @@
*.pyc
build/
dist/
+.tox/
paramiko.egg-info/
test.log
+docs/
diff --git a/.travis.yml b/.travis.yml
index 312a1846..6896b897 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -9,6 +9,6 @@ install:
script: python test.py
notifications:
irc:
- channels: "irc.freenode.org#fabric"
+ channels: "irc.freenode.org#paramiko"
on_success: change
on_failure: change
diff --git a/Makefile b/Makefile
index 3c12990c..572f867a 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
release: docs
python setup.py sdist register upload
-docs:
+docs: paramiko/*
epydoc --no-private -o docs/ paramiko
clean:
diff --git a/NEWS b/NEWS
index 37739c60..7b45d621 100644
--- a/NEWS
+++ b/NEWS
@@ -12,9 +12,67 @@ Issues noted as "Fabric #NN" can be found at https://github.com/fabric/fabric/.
Releases
========
-v1.8.0 <DATE>
----------------
+v1.10.0 (DD MM YYYY)
+--------------------
+
+* #110: Honor SSH config `AddressFamily` setting when looking up local
+ host's FQDN. Thanks to John Hensley for the patch.
+* #128: Defer FQDN resolution until needed, when parsing SSH config files.
+ Thanks to Parantapa Bhattacharya for catch & patch.
+* #102: Forego random padding for packets when running under `*-ctr` ciphers.
+ This corrects some slowdowns on platforms where random byte generation is
+ inefficient (e.g. Windows). Thanks to `@warthog618` for catch & patch, and
+ Michael van der Kolff for code/technique review.
+* #127: Turn `SFTPFile` into a context manager. Thanks to Michael Williamson
+* for the patch.
+* #116: Limit `Message.get_bytes` to an upper bound of 1MB to protect against
+ potential DoS vectors. Thanks to `@mvschaik` for catch & patch.
+* #115: Add convenience `get_pty` kwarg to `Client.exec_command` so users not
+ manually controlling a channel object can still toggle PTY creation. Thanks
+ to Michael van der Kolff for the patch.
+* #71: Add `SFTPClient.putfo` and `.getfo` methods to allow direct
+ uploading/downloading of file-like objects. Thanks to Eric Buehl for the
+ patch.
+* #113: Add `timeout` parameter to `SSHClient.exec_command` for easier setting
+ of the command's internal channel object's timeout. Thanks to Cernov Vladimir
+ for the patch.
+* #94: Remove duplication of SSH port constant. Thanks to Olle Lundberg for the
+ catch.
+* #80: Expose the internal "is closed" property of the file transfer class
+ `BufferedFile` as `.closed`, better conforming to Python's file interface.
+ Thanks to `@smunaut` and James Hiscock for catch & patch.
+
+v1.9.0 (6th Nov 2012)
+---------------------
+
+* #97 (with a little #93): Improve config parsing of `ProxyCommand` directives
+ and provide a wrapper class to allow subprocess-driven proxy commands to be
+ used as `sock=` arguments for `SSHClient.connect`.
+* #77: Allow `SSHClient.connect()` to take an explicit `sock` parameter
+ overriding creation of an internal, implicit socket object.
+* Thanks in no particular order to Erwin Bolwidt, Oskari Saarenmaa, Steven
+ Noonan, Vladimir Lazarenko, Lincoln de Sousa, Valentino Volonghi, Olle
+ Lundberg, and Github user `@acrish` for the various and sundry patches
+ leading to the above changes.
+
+v1.8.1 (6th Nov 2012)
+---------------------
+
+* #90: Ensure that callbacks handed to `SFTPClient.get()` always fire at least
+ once, even for zero-length files downloaded. Thanks to Github user `@enB` for
+ the catch.
+* #85: Paramiko's test suite overrides
+ `unittest.TestCase.assertTrue/assertFalse` to provide these modern assertions
+ to Python 2.2/2.3, which lacked them. However on newer Pythons such as 2.7,
+ this now causes deprecation warnings. The overrides have been patched to only
+ execute when necessary. Thanks to `@Arfrever` for catch & patch.
+
+
+v1.8.0 (3rd Oct 2012)
+---------------------
+* #17 ('ssh' 28): Fix spurious `NoneType has no attribute 'error'` and similar
+ exceptions that crop up on interpreter exit.
* 'ssh' 32: Raise a more useful error explaining which `known_hosts` key line was
problematic, when encountering `binascii` issues decoding known host keys.
Thanks to `@thomasvs` for catch & patch.
diff --git a/README b/README
index 9e9cc3a7..310a7f02 100644
--- a/README
+++ b/README
@@ -5,7 +5,7 @@ paramiko
:Paramiko: Python SSH module
:Copyright: Copyright (c) 2003-2009 Robey Pointer <robeypointer@gmail.com>
-:Copyright: Copyright (c) 2012 Jeff Forcier <jeff@bitprophet.org>
+:Copyright: Copyright (c) 2013 Jeff Forcier <jeff@bitprophet.org>
:License: LGPL
:Homepage: https://github.com/paramiko/paramiko/
@@ -20,7 +20,7 @@ What
----
"paramiko" is a combination of the esperanto words for "paranoid" and
-"friend". it's a module for python 2.2+ that implements the SSH2 protocol
+"friend". it's a module for python 2.5+ that implements the SSH2 protocol
for secure (encrypted and authenticated) connections to remote machines.
unlike SSL (aka TLS), SSH2 protocol does not require hierarchical
certificates signed by a powerful central authority. you may know SSH2 as
@@ -39,8 +39,7 @@ that should have come with this archive.
Requirements
------------
- - python 2.3 or better <http://www.python.org/>
- (python 2.2 is also supported, but not recommended)
+ - python 2.5 or better <http://www.python.org/>
- pycrypto 2.1 or better <https://www.dlitz.net/software/pycrypto/>
If you have setuptools, you can build and install paramiko and all its
@@ -58,19 +57,6 @@ should also work on Windows, though i don't test it as frequently there.
if you run into Windows problems, send me a patch: portability is important
to me.
-python 2.2 may work, thanks to some patches from Roger Binns. things to
-watch out for:
-
- * sockets in 2.2 don't support timeouts, so the 'select' module is
- imported to do polling.
- * logging is mostly stubbed out. it works just enough to let paramiko
- create log files for debugging, if you want them. to get real logging,
- you can backport python 2.3's logging package. Roger has done that
- already:
- http://sourceforge.net/project/showfiles.php?group_id=75211&package_id=113804
-
-you really should upgrade to python 2.3. laziness is no excuse! :)
-
some python distributions don't include the utf-8 string encodings, for
reasons of space (misdirected as that is). if your distribution is
missing encodings, you'll see an error like this::
diff --git a/fabfile.py b/fabfile.py
new file mode 100644
index 00000000..29394f94
--- /dev/null
+++ b/fabfile.py
@@ -0,0 +1,13 @@
+from fabric.api import task, sudo, env
+from fabric.contrib.project import rsync_project
+
+
+@task
+def upload_docs():
+ target = "/var/www/paramiko.org"
+ staging = "/tmp/paramiko_docs"
+ sudo("mkdir -p %s" % staging)
+ sudo("chown -R %s %s" % (env.user, staging))
+ sudo("rm -rf %s/*" % target)
+ rsync_project(local_dir='docs/', remote_dir=staging, delete=True)
+ sudo("cp -R %s/* %s/" % (staging, target))
diff --git a/paramiko/__init__.py b/paramiko/__init__.py
index 8c158538..e2b359fb 100644
--- a/paramiko/__init__.py
+++ b/paramiko/__init__.py
@@ -18,7 +18,7 @@
"""
I{Paramiko} (a combination of the esperanto words for "paranoid" and "friend")
-is a module for python 2.3 or greater that implements the SSH2 protocol for
+is a module for python 2.5 or greater that implements the SSH2 protocol for
secure (encrypted and authenticated) connections to remote machines. Unlike
SSL (aka TLS), the SSH2 protocol does not require hierarchical certificates
signed by a powerful central authority. You may know SSH2 as the protocol that
@@ -50,12 +50,12 @@ Website: U{https://github.com/paramiko/paramiko/}
import sys
-if sys.version_info < (2, 2):
- raise RuntimeError('You need python 2.2 for this module.')
+if sys.version_info < (2, 5):
+ raise RuntimeError('You need python 2.5+ for this module.')
__author__ = "Jeff Forcier <jeff@bitprophet.org>"
-__version__ = "1.8.0"
+__version__ = "1.10.0"
__license__ = "GNU Lesser General Public License (LGPL)"
@@ -65,7 +65,7 @@ from auth_handler import AuthHandler
from channel import Channel, ChannelFile
from ssh_exception import SSHException, PasswordRequiredException, \
BadAuthenticationType, ChannelException, BadHostKeyException, \
- AuthenticationException
+ AuthenticationException, ProxyCommandFailure
from server import ServerInterface, SubsystemHandler, InteractiveQuery
from rsakey import RSAKey
from dsskey import DSSKey
@@ -83,6 +83,7 @@ from agent import Agent, AgentKey
from pkey import PKey
from hostkeys import HostKeys
from config import SSHConfig
+from proxy import ProxyCommand
# fix module names for epydoc
for c in locals().values():
@@ -119,6 +120,8 @@ __all__ = [ 'Transport',
'BadAuthenticationType',
'ChannelException',
'BadHostKeyException',
+ 'ProxyCommand',
+ 'ProxyCommandFailure',
'SFTP',
'SFTPFile',
'SFTPHandle',
diff --git a/paramiko/client.py b/paramiko/client.py
index 0f408977..f8638068 100644
--- a/paramiko/client.py
+++ b/paramiko/client.py
@@ -28,6 +28,7 @@ import warnings
from paramiko.agent import Agent
from paramiko.common import *
+from paramiko.config import SSH_PORT
from paramiko.dsskey import DSSKey
from paramiko.hostkeys import HostKeys
from paramiko.resource import ResourceManager
@@ -37,8 +38,6 @@ from paramiko.transport import Transport
from paramiko.util import retry_on_signal
-SSH_PORT = 22
-
class MissingHostKeyPolicy (object):
"""
Interface for defining the policy that L{SSHClient} should use when the
@@ -229,7 +228,7 @@ class SSHClient (object):
def connect(self, hostname, port=SSH_PORT, username=None, password=None, pkey=None,
key_filename=None, timeout=None, allow_agent=True, look_for_keys=True,
- compress=False):
+ compress=False, sock=None):
"""
Connect to an SSH server and authenticate to it. The server's host key
is checked against the system host keys (see L{load_system_host_keys})
@@ -272,6 +271,9 @@ class SSHClient (object):
@type look_for_keys: bool
@param compress: set to True to turn on compression
@type compress: bool
+ @param sock: an open socket or socket-like object (such as a
+ L{Channel}) to use for communication to the target host
+ @type sock: socket
@raise BadHostKeyException: if the server's host key could not be
verified
@@ -280,21 +282,23 @@ class SSHClient (object):
establishing an SSH session
@raise socket.error: if a socket error occurred while connecting
"""
- for (family, socktype, proto, canonname, sockaddr) in socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM):
- if socktype == socket.SOCK_STREAM:
- af = family
- addr = sockaddr
- break
- else:
- # some OS like AIX don't indicate SOCK_STREAM support, so just guess. :(
- af, _, _, _, addr = socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
- sock = socket.socket(af, socket.SOCK_STREAM)
- if timeout is not None:
- try:
- sock.settimeout(timeout)
- except:
- pass
- retry_on_signal(lambda: sock.connect(addr))
+ if not sock:
+ for (family, socktype, proto, canonname, sockaddr) in socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM):
+ if socktype == socket.SOCK_STREAM:
+ af = family
+ addr = sockaddr
+ break
+ else:
+ # some OS like AIX don't indicate SOCK_STREAM support, so just guess. :(
+ af, _, _, _, addr = socket.getaddrinfo(hostname, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
+ sock = socket.socket(af, socket.SOCK_STREAM)
+ if timeout is not None:
+ try:
+ sock.settimeout(timeout)
+ except:
+ pass
+ retry_on_signal(lambda: sock.connect(addr))
+
t = self._transport = Transport(sock)
t.use_compression(compress=compress)
if self._log_channel is not None:
@@ -345,7 +349,7 @@ class SSHClient (object):
self._agent.close()
self._agent = None
- def exec_command(self, command, bufsize=-1):
+ def exec_command(self, command, bufsize=-1, timeout=None, get_pty=False):
"""
Execute a command on the SSH server. A new L{Channel} is opened and
the requested command is executed. The command's input and output
@@ -356,12 +360,17 @@ class SSHClient (object):
@type command: str
@param bufsize: interpreted the same way as by the built-in C{file()} function in python
@type bufsize: int
+ @param timeout: set command's channel timeout. See L{Channel.settimeout}.settimeout
+ @type timeout: int
@return: the stdin, stdout, and stderr of the executing command
@rtype: tuple(L{ChannelFile}, L{ChannelFile}, L{ChannelFile})
@raise SSHException: if the server fails to execute the command
"""
chan = self._transport.open_session()
+ if(get_pty):
+ chan.get_pty()
+ chan.settimeout(timeout)
chan.exec_command(command)
stdin = chan.makefile('wb', bufsize)
stdout = chan.makefile('rb', bufsize)
diff --git a/paramiko/config.py b/paramiko/config.py
index c5a58040..8e45ad55 100644
--- a/paramiko/config.py
+++ b/paramiko/config.py
@@ -27,8 +27,53 @@ import re
import socket
SSH_PORT = 22
+proxy_re = re.compile(r"^(proxycommand)\s*=*\s*(.*)", re.I)
+class LazyFqdn(object):
+ """
+ Returns the host's fqdn on request as string.
+ """
+
+ def __init__(self, config):
+ self.fqdn = None
+ self.config = config
+
+ def __str__(self):
+ if self.fqdn is None:
+ #
+ # If the SSH config contains AddressFamily, use that when
+ # determining the local host's FQDN. Using socket.getfqdn() from
+ # the standard library is the most general solution, but can
+ # result in noticeable delays on some platforms when IPv6 is
+ # misconfigured or not available, as it calls getaddrinfo with no
+ # address family specified, so both IPv4 and IPv6 are checked.
+ #
+
+ # Handle specific option
+ fqdn = None
+ address_family = self.config.get('addressfamily', 'any').lower()
+ if address_family != 'any':
+ family = socket.AF_INET if address_family == 'inet' \
+ else socket.AF_INET6
+ results = socket.getaddrinfo(host,
+ None,
+ family,
+ socket.SOCK_DGRAM,
+ socket.IPPROTO_IP,
+ socket.AI_CANONNAME)
+ for res in results:
+ af, socktype, proto, canonname, sa = res
+ if canonname and '.' in canonname:
+ fqdn = canonname
+ break
+ # Handle 'any' / unspecified
+ if fqdn is None:
+ fqdn = socket.getfqdn()
+ # Cache
+ self.fqdn = fqdn
+ return self.fqdn
+
class SSHConfig (object):
"""
Representation of config information as stored in the format used by
@@ -44,7 +89,6 @@ class SSHConfig (object):
"""
Create a new OpenSSH config object.
"""
- self._proxyregex = re.compile(r"^(proxycommand)\s*=*\s*(.*)", re.I)
self._config = []
def parse(self, file_obj):
@@ -60,15 +104,13 @@ class SSHConfig (object):
if (line == '') or (line[0] == '#'):
continue
if '=' in line:
- if not line.lower().startswith('proxycommand'):
+ # Ensure ProxyCommand gets properly split
+ if line.lower().strip().startswith('proxycommand'):
+ match = proxy_re.match(line)
+ key, value = match.group(1).lower(), match.group(2)
+ else:
key, value = line.split('=', 1)
key = key.strip().lower()
- else:
- #ProxyCommand have been specified with an equal
- # sign. Eat that and split in two groups.
- match = self._proxyregex.match(line)
- key = match.group(1).lower()
- value = match.group(2)
else:
# find first whitespace, and split there
i = 0
@@ -172,7 +214,7 @@ class SSHConfig (object):
remoteuser = user
host = socket.gethostname().split('.')[0]
- fqdn = socket.getfqdn()
+ fqdn = LazyFqdn(config)
homedir = os.path.expanduser('~')
replacements = {'controlpath':
[
@@ -200,6 +242,7 @@ class SSHConfig (object):
('%r', remoteuser)
]
}
+
for k in config:
if k in replacements:
for find, replace in replacements[k]:
diff --git a/paramiko/file.py b/paramiko/file.py
index d4aec8e3..7e2904e1 100644
--- a/paramiko/file.py
+++ b/paramiko/file.py
@@ -354,6 +354,10 @@ class BufferedFile (object):
"""
return self
+ @property
+ def closed(self):
+ return self._closed
+
### overrides...
diff --git a/paramiko/message.py b/paramiko/message.py
index 366c43c9..47acc34b 100644
--- a/paramiko/message.py
+++ b/paramiko/message.py
@@ -110,7 +110,8 @@ class Message (object):
@rtype: string
"""
b = self.packet.read(n)
- if len(b) < n:
+ max_pad_size = 1<<20 # Limit padding to 1 MB
+ if len(b) < n and n < max_pad_size:
return b + '\x00' * (n - len(b))
return b
diff --git a/paramiko/packet.py b/paramiko/packet.py
index 97820619..38a6d4b5 100644
--- a/paramiko/packet.py
+++ b/paramiko/packet.py
@@ -29,7 +29,7 @@ import time
from paramiko.common import *
from paramiko import util
-from paramiko.ssh_exception import SSHException
+from paramiko.ssh_exception import SSHException, ProxyCommandFailure
from paramiko.message import Message
@@ -87,6 +87,7 @@ class Packetizer (object):
self.__mac_size_in = 0
self.__block_engine_out = None
self.__block_engine_in = None
+ self.__sdctr_out = False
self.__mac_engine_out = None
self.__mac_engine_in = None
self.__mac_key_out = ''
@@ -110,11 +111,12 @@ class Packetizer (object):
"""
self.__logger = log
- def set_outbound_cipher(self, block_engine, block_size, mac_engine, mac_size, mac_key):
+ def set_outbound_cipher(self, block_engine, block_size, mac_engine, mac_size, mac_key, sdctr=False):
"""
Switch outbound data cipher.
"""
self.__block_engine_out = block_engine
+ self.__sdctr_out = sdctr
self.__block_size_out = block_size
self.__mac_engine_out = mac_engine
self.__mac_size_out = mac_size
@@ -254,6 +256,8 @@ class Packetizer (object):
retry_write = True
else:
n = -1
+ except ProxyCommandFailure:
+ raise # so it doesn't get swallowed by the below catchall
except Exception:
# could be: (32, 'Broken pipe')
n = -1
@@ -488,12 +492,12 @@ class Packetizer (object):
padding = 3 + bsize - ((len(payload) + 8) % bsize)
packet = struct.pack('>IB', len(payload) + padding + 1, padding)
packet += payload
- if self.__block_engine_out is not None:
- packet += rng.read(padding)
- else:
- # cute trick i caught openssh doing: if we're not encrypting,
+ if self.__sdctr_out or self.__block_engine_out is None:
+ # cute trick i caught openssh doing: if we're not encrypting or SDCTR mode (RFC4344),
# don't waste random bytes for the padding
packet += (chr(0) * padding)
+ else:
+ packet += rng.read(padding)
return packet
def _trigger_rekey(self):
diff --git a/paramiko/proxy.py b/paramiko/proxy.py
new file mode 100644
index 00000000..218b76e2
--- /dev/null
+++ b/paramiko/proxy.py
@@ -0,0 +1,91 @@
+# Copyright (C) 2012 Yipit, Inc <coders@yipit.com>
+#
+# This file is part of paramiko.
+#
+# Paramiko is free software; you can redistribute it and/or modify it under the
+# terms of the GNU Lesser General Public License as published by the Free
+# Software Foundation; either version 2.1 of the License, or (at your option)
+# any later version.
+#
+# Paramiko is distrubuted in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+# A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with Paramiko; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
+
+"""
+L{ProxyCommand}.
+"""
+
+import os
+from shlex import split as shlsplit
+import signal
+from subprocess import Popen, PIPE
+
+from paramiko.ssh_exception import ProxyCommandFailure
+
+
+class ProxyCommand(object):
+ """
+ Wraps a subprocess running ProxyCommand-driven programs.
+
+ This class implements a the socket-like interface needed by the
+ L{Transport} and L{Packetizer} classes. Using this class instead of a
+ regular socket makes it possible to talk with a Popen'd command that will
+ proxy traffic between the client and a server hosted in another machine.
+ """
+ def __init__(self, command_line):
+ """
+ Create a new CommandProxy instance. The instance created by this
+ class can be passed as an argument to the L{Transport} class.
+
+ @param command_line: the command that should be executed and
+ used as the proxy.
+ @type command_line: str
+ """
+ self.cmd = shlsplit(command_line)
+ self.process = Popen(self.cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
+
+ def send(self, content):
+ """
+ Write the content received from the SSH client to the standard
+ input of the forked command.
+
+ @param content: string to be sent to the forked command
+ @type content: str
+ """
+ try:
+ self.process.stdin.write(content)
+ except IOError, e:
+ # There was a problem with the child process. It probably
+ # died and we can't proceed. The best option here is to
+ # raise an exception informing the user that the informed
+ # ProxyCommand is not working.
+ raise BadProxyCommand(' '.join(self.cmd), e.strerror)
+ return len(content)
+
+ def recv(self, size):
+ """
+ Read from the standard output of the forked program.
+
+ @param size: how many chars should be read
+ @type size: int
+
+ @return: the length of the read content
+ @rtype: int
+ """
+ try:
+ return os.read(self.process.stdout.fileno(), size)
+ except IOError, e:
+ raise BadProxyCommand(' '.join(self.cmd), e.strerror)
+
+ def close(self):
+ os.kill(self.process.pid, signal.SIGTERM)
+
+ def settimeout(self, timeout):
+ # Timeouts are meaningless for this implementation, but are part of the
+ # spec, so must be present.
+ pass
diff --git a/paramiko/sftp_client.py b/paramiko/sftp_client.py
index f446ba3d..7df643f5 100644
--- a/paramiko/sftp_client.py
+++ b/paramiko/sftp_client.py
@@ -198,7 +198,7 @@ class SFTPClient (BaseSFTP):
Open a file on the remote server. The arguments are the same as for
python's built-in C{file} (aka C{open}). A file-like object is
returned, which closely mimics the behavior of a normal python file
- object.
+ object, including the ability to be used as a context manager.
The mode indicates how the file is to be opened: C{'r'} for reading,
C{'w'} for writing (truncating an existing file), C{'a'} for appending,
@@ -533,6 +533,56 @@ class SFTPClient (BaseSFTP):
"""
return self._cwd
+ def putfo(self, fl, remotepath, file_size=0, callback=None, confirm=True):
+ """
+ Copy the contents of an open file object (C{fl}) to the SFTP server as
+ C{remotepath}. Any exception raised by operations will be passed through.
+
+ The SFTP operations use pipelining for speed.
+
+ @param fl: opened file or file-like object to copy
+ @type localpath: object
+ @param remotepath: the destination path on the SFTP server
+ @type remotepath: str
+ @param file_size: optional size parameter passed to callback. If none is
+ specified, size defaults to 0
+ @type file_size: int
+ @param callback: optional callback function that accepts the bytes
+ transferred so far and the total bytes to be transferred
+ (since 1.7.4)
+ @type callback: function(int, int)
+ @param confirm: whether to do a stat() on the file afterwards to
+ confirm the file size (since 1.7.7)
+ @type confirm: bool
+
+ @return: an object containing attributes about the given file
+ (since 1.7.4)
+ @rtype: SFTPAttributes
+
+ @since: 1.4
+ """
+ fr = self.file(remotepath, 'wb')
+ fr.set_pipelined(True)
+ size = 0
+ try:
+ while True:
+ data = fl.read(32768)
+ fr.write(data)
+ size += len(data)
+ if callback is not None:
+ callback(size, file_size)
+ if len(data) == 0:
+ break
+ finally:
+ fr.close()
+ if confirm and file_size:
+ s = self.stat(remotepath)
+ if s.st_size != size:
+ raise IOError('size mismatch in put! %d != %d' % (s.st_size, size))
+ else:
+ s = SFTPAttributes()
+ return s
+
def put(self, localpath, remotepath, callback=None, confirm=True):
"""
Copy a local file (C{localpath}) to the SFTP server as C{remotepath}.
@@ -562,29 +612,46 @@ class SFTPClient (BaseSFTP):
file_size = os.stat(localpath).st_size
fl = file(localpath, 'rb')
try:
- fr = self.file(remotepath, 'wb')
- fr.set_pipelined(True)
- size = 0
- try:
- while True:
- data = fl.read(32768)
- if len(data) == 0:
- break
- fr.write(data)
- size += len(data)
- if callback is not None:
- callback(size, file_size)
- finally:
- fr.close()
+ return self.putfo(fl, remotepath, os.stat(localpath).st_size, callback, confirm)
finally:
fl.close()
- if confirm:
- s = self.stat(remotepath)
- if s.st_size != size:
- raise IOError('size mismatch in put! %d != %d' % (s.st_size, size))
- else:
- s = SFTPAttributes()
- return s
+
+ def getfo(self, remotepath, fl, callback=None):
+ """
+ Copy a remote file (C{remotepath}) from the SFTP server and write to
+ an open file or file-like object, C{fl}. Any exception raised by
+ operations will be passed through. This method is primarily provided
+ as a convenience.
+
+ @param remotepath: opened file or file-like object to copy to
+ @type remotepath: object
+ @param fl: the destination path on the local host or open file
+ object
+ @type localpath: str
+ @param callback: optional callback function that accepts the bytes
+ transferred so far and the total bytes to be transferred
+ (since 1.7.4)
+ @type callback: function(int, int)
+ @return: the number of bytes written to the opened file object
+
+ @since: 1.4
+ """
+ fr = self.file(remotepath, 'rb')
+ file_size = self.stat(remotepath).st_size
+ fr.prefetch()
+ try:
+ size = 0
+ while True:
+ data = fr.read(32768)
+ fl.write(data)
+ size += len(data)
+ if callback is not None:
+ callback(size, file_size)
+ if len(data) == 0:
+ break
+ finally:
+ fr.close()
+ return size
def get(self, remotepath, localpath, callback=None):
"""
@@ -603,25 +670,12 @@ class SFTPClient (BaseSFTP):
@since: 1.4
"""
- fr = self.file(remotepath, 'rb')
file_size = self.stat(remotepath).st_size
- fr.prefetch()
+ fl = file(localpath, 'wb')
try:
- fl = file(localpath, 'wb')
- try:
- size = 0
- while True:
- data = fr.read(32768)
- if len(data) == 0:
- break
- fl.write(data)
- size += len(data)
- if callback is not None:
- callback(size, file_size)
- finally:
- fl.close()
+ size = self.getfo(remotepath, fl, callback)
finally:
- fr.close()
+ fl.close()
s = os.stat(localpath)
if s.st_size != size:
raise IOError('size mismatch in get! %d != %d' % (s.st_size, size))
diff --git a/paramiko/sftp_file.py b/paramiko/sftp_file.py
index 8c5c7aca..d4ecb89f 100644
--- a/paramiko/sftp_file.py
+++ b/paramiko/sftp_file.py
@@ -34,6 +34,9 @@ from paramiko.sftp_attr import SFTPAttributes
class SFTPFile (BufferedFile):
"""
Proxy object for a file on the remote server, in client mode SFTP.
+
+ Instances of this class may be used as context managers in the same way
+ that built-in Python file objects are.
"""
# Some sftp servers will choke if you send read/write requests larger than
@@ -474,3 +477,9 @@ class SFTPFile (BufferedFile):
x = self._saved_exception
self._saved_exception = None
raise x
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, type, value, traceback):
+ self.close()
diff --git a/paramiko/ssh_exception.py b/paramiko/ssh_exception.py
index 68924d0f..f2406dcf 100644
--- a/paramiko/ssh_exception.py
+++ b/paramiko/ssh_exception.py
@@ -113,3 +113,20 @@ class BadHostKeyException (SSHException):
self.key = got_key
self.expected_key = expected_key
+
+class ProxyCommandFailure (SSHException):
+ """
+ The "ProxyCommand" found in the .ssh/config file returned an error.
+
+ @ivar command: The command line that is generating this exception.
+ @type command: str
+ @ivar error: The error captured from the proxy command output.
+ @type error: str
+ """
+ def __init__(self, command, error):
+ SSHException.__init__(self,
+ '"ProxyCommand (%s)" returned non-zero exit status: %s' % (
+ command, error
+ )
+ )
+ self.error = error
diff --git a/paramiko/transport.py b/paramiko/transport.py
index 04680a9f..fd6dab76 100644
--- a/paramiko/transport.py
+++ b/paramiko/transport.py
@@ -44,7 +44,8 @@ from paramiko.primes import ModulusPack
from paramiko.rsakey import RSAKey
from paramiko.server import ServerInterface
from paramiko.sftp_client import SFTPClient
-from paramiko.ssh_exception import SSHException, BadAuthenticationType, ChannelException
+from paramiko.ssh_exception import (SSHException, BadAuthenticationType,
+ ChannelException, ProxyCommandFailure)
from paramiko.util import retry_on_signal
from Crypto import Random
@@ -1674,6 +1675,8 @@ class Transport (threading.Thread):
timeout = 2
try:
buf = self.packetizer.readline(timeout)
+ except ProxyCommandFailure:
+ raise
except Exception, x:
raise SSHException('Error reading SSH protocol banner' + str(x))
if buf[:4] == 'SSH-':
@@ -1882,7 +1885,8 @@ class Transport (threading.Thread):
mac_key = self._compute_key('F', mac_engine.digest_size)
else:
mac_key = self._compute_key('E', mac_engine.digest_size)
- self.packetizer.set_outbound_cipher(engine, block_size, mac_engine, mac_size, mac_key)
+ sdctr = self.local_cipher.endswith('-ctr')
+ self.packetizer.set_outbound_cipher(engine, block_size, mac_engine, mac_size, mac_key, sdctr)
compress_out = self._compression_info[self.local_compression][0]
if (compress_out is not None) and ((self.local_compression != 'zlib@openssh.com') or self.authenticated):
self._log(DEBUG, 'Switching on outbound compression ...')
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 00000000..75112a23
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,2 @@
+pycrypto
+tox
diff --git a/setup.py b/setup.py
index 73407b03..9812a4f4 100644
--- a/setup.py
+++ b/setup.py
@@ -52,7 +52,7 @@ if sys.platform == 'darwin':
setup(name = "paramiko",
- version = "1.8.0",
+ version = "1.10.0",
description = "SSH2 protocol library",
author = "Jeff Forcier",
author_email = "jeff@bitprophet.org",
diff --git a/tests/test_buffered_pipe.py b/tests/test_buffered_pipe.py
index f285d05b..b9d91f6a 100644
--- a/tests/test_buffered_pipe.py
+++ b/tests/test_buffered_pipe.py
@@ -26,6 +26,8 @@ import unittest
from paramiko.buffered_pipe import BufferedPipe, PipeTimeout
from paramiko import pipe
+from util import ParamikoTest
+
def delay_thread(pipe):
pipe.feed('a')
@@ -39,11 +41,7 @@ def close_thread(pipe):
pipe.close()
-class BufferedPipeTest (unittest.TestCase):
-
- assertTrue = unittest.TestCase.failUnless # for Python 2.3 and below
- assertFalse = unittest.TestCase.failIf # for Python 2.3 and below
-
+class BufferedPipeTest(ParamikoTest):
def test_1_buffered_pipe(self):
p = BufferedPipe()
self.assert_(not p.read_ready())
diff --git a/tests/test_sftp.py b/tests/test_sftp.py
index 2eadabcd..f95da69c 100755
--- a/tests/test_sftp.py
+++ b/tests/test_sftp.py
@@ -23,6 +23,8 @@ a real actual sftp server is contacted, and a new folder is created there to
do test file operations in (so no existing files will be harmed).
"""
+from __future__ import with_statement
+
from binascii import hexlify
import logging
import os
@@ -188,6 +190,17 @@ class SFTPTest (unittest.TestCase):
finally:
sftp.remove(FOLDER + '/duck.txt')
+ def test_3_sftp_file_can_be_used_as_context_manager(self):
+ """
+ verify that an opened file can be used as a context manager
+ """
+ try:
+ with sftp.open(FOLDER + '/duck.txt', 'w') as f:
+ f.write(ARTICLE)
+ self.assertEqual(sftp.stat(FOLDER + '/duck.txt').st_size, 1483)
+ finally:
+ sftp.remove(FOLDER + '/duck.txt')
+
def test_4_append(self):
"""
verify that a file can be opened for append, and tell() still works.
diff --git a/tests/test_transport.py b/tests/test_transport.py
index cea4a1dd..1c57d18d 100644
--- a/tests/test_transport.py
+++ b/tests/test_transport.py
@@ -36,6 +36,7 @@ from paramiko import OPEN_SUCCEEDED, OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
from paramiko.common import MSG_KEXINIT, MSG_CHANNEL_WINDOW_ADJUST
from paramiko.message import Message
from loop import LoopSocket
+from util import ParamikoTest
LONG_BANNER = """\
@@ -105,11 +106,7 @@ class NullServer (ServerInterface):
return OPEN_SUCCEEDED
-class TransportTest (unittest.TestCase):
-
- assertTrue = unittest.TestCase.failUnless # for Python 2.3 and below
- assertFalse = unittest.TestCase.failIf # for Python 2.3 and below
-
+class TransportTest(ParamikoTest):
def setUp(self):
self.socks = LoopSocket()
self.sockc = LoopSocket()
diff --git a/tests/test_util.py b/tests/test_util.py
index 9a155bfa..efda9b2f 100644
--- a/tests/test_util.py
+++ b/tests/test_util.py
@@ -27,7 +27,9 @@ import os
import unittest
from Crypto.Hash import SHA
import paramiko.util
+from paramiko.util import lookup_ssh_host_config as host_config
+from util import ParamikoTest
test_config_file = """\
Host *
@@ -58,17 +60,7 @@ BGQ3GQ/Fc7SX6gkpXkwcZryoi4kNFhHu5LvHcZPdxXV1D+uTMfGS1eyd2Yz/DoNWXNAl8TI0cAsW\
from paramiko import *
-class UtilTest (unittest.TestCase):
-
- assertTrue = unittest.TestCase.failUnless # for Python 2.3 and below
- assertFalse = unittest.TestCase.failIf # for Python 2.3 and below
-
- def setUp(self):
- pass
-
- def tearDown(self):
- pass
-
+class UtilTest(ParamikoTest):
def test_1_import(self):
"""
verify that all the classes can be imported from paramiko.
@@ -213,7 +205,52 @@ Host *
self.assertRaises(AssertionError,
lambda: paramiko.util.retry_on_signal(raises_other_exception))
- def test_9_host_config_test_negation(self):
+ def test_9_proxycommand_config_equals_parsing(self):
+ """
+ ProxyCommand should not split on equals signs within the value.
+ """
+ conf = """
+Host space-delimited
+ ProxyCommand foo bar=biz baz
+
+Host equals-delimited
+ ProxyCommand=foo bar=biz baz
+"""
+ f = cStringIO.StringIO(conf)
+ config = paramiko.util.parse_ssh_config(f)
+ for host in ('space-delimited', 'equals-delimited'):
+ self.assertEquals(
+ host_config(host, config)['proxycommand'],
+ 'foo bar=biz baz'
+ )
+
+ def test_10_proxycommand_interpolation(self):
+ """
+ ProxyCommand should perform interpolation on the value
+ """
+ config = paramiko.util.parse_ssh_config(cStringIO.StringIO("""
+Host specific
+ Port 37
+ ProxyCommand host %h port %p lol
+
+Host portonly
+ Port 155
+
+Host *
+ Port 25
+ ProxyCommand host %h port %p
+"""))
+ for host, val in (
+ ('foo.com', "host foo.com port 25"),
+ ('specific', "host specific port 37 lol"),
+ ('portonly', "host portonly port 155"),
+ ):
+ self.assertEquals(
+ host_config(host, config)['proxycommand'],
+ val
+ )
+
+ def test_11_host_config_test_negation(self):
test_config_file = """
Host www13.* !*.example.com
Port 22
@@ -235,7 +272,7 @@ Host *
{'hostname': host, 'port': '8080'}
)
- def test_10_host_config_test_proxycommand(self):
+ def test_12_host_config_test_proxycommand(self):
test_config_file = """
Host proxy-with-equal-divisor-and-space
ProxyCommand = foo=bar
diff --git a/tests/util.py b/tests/util.py
new file mode 100644
index 00000000..2e0be087
--- /dev/null
+++ b/tests/util.py
@@ -0,0 +1,10 @@
+import unittest
+
+
+class ParamikoTest(unittest.TestCase):
+ # for Python 2.3 and below
+ if not hasattr(unittest.TestCase, 'assertTrue'):
+ assertTrue = unittest.TestCase.failUnless
+ if not hasattr(unittest.TestCase, 'assertFalse'):
+ assertFalse = unittest.TestCase.failIf
+
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 00000000..6cb80012
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,6 @@
+[tox]
+envlist = py25,py26,py27
+
+[testenv]
+commands = pip install --use-mirrors -q -r requirements.txt
+ python test.py