-
- Notifications
You must be signed in to change notification settings - Fork 19.4k
Closed
Labels
IO CSVread_csv, to_csvread_csv, to_csvTestingpandas testing functions or related to the test suitepandas testing functions or related to the test suite
Milestone
Description
#17057 fixes all but this warning. I guess something is not getting closed in s3fs.
(pandas) bash-3.2$ pytest pandas/tests/io/parser/ =========================================================================================== test session starts =========================================================================================== platform darwin -- Python 3.6.1, pytest-3.1.2, py-1.4.34, pluggy-0.4.0 rootdir: /Users/jreback/pandas, inifile: setup.cfg plugins: cov-2.3.1, xdist-1.16.0 collected 837 items pandas/tests/io/parser/test_network.py .......................... pandas/tests/io/parser/test_parsers.py .......................s............................................................................s...............................................................................................s....................................................s.............................s............................................................................s...............................................................................................s....................................................s..................................................................................s...............................................................................................s...............................................................s...... pandas/tests/io/parser/test_read_fwf.py ........................ pandas/tests/io/parser/test_textreader.py ......................... pandas/tests/io/parser/test_unsupported.py ....... ============================================================================================ warnings summary ============================================================================================= pandas/tests/io/parser/test_network.py::TestS3::()::test_s3_fails /Users/jreback/pandas/pandas/tests/io/parser/test_network.py:174: ResourceWarning: unclosed <ssl.SSLSocket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.15', 60648), raddr=('52.216.32.16', 443)> read_csv('s3://nyqpug/asdf.csv') -- Docs: http://doc.pytest.org/en/latest/warnings.html =========================================================================== 826 passed, 11 skipped, 1 warnings in 65.08 seconds =========================================================================== This is doing this internally.
In [7]: import s3fs In [8]: s3fs.__version__ Out[8]: '0.1.1' In [5]: fs = s3fs.S3FileSystem(anon=True) In [6]: fs.open('nyqpug/asdf.csv') --------------------------------------------------------------------------- ClientError Traceback (most recent call last) /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, path, refresh, **kwargs) 393 out = self._call_s3(self.s3.head_object, --> 394 kwargs, Bucket=bucket, Key=key, **self.req_kw) 395 out = {'ETag': out['ETag'], 'Key': '/'.join([bucket, key]), /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in _call_s3(self, method, *akwarglist, **kwargs) 167 additional_kwargs.update(kwargs) --> 168 return method(**additional_kwargs) 169 /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs) 252 # The "self" in this scope is referring to the BaseClient. --> 253 return self._make_api_call(operation_name, kwargs) 254 /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params) 543 error_class = self.exceptions.from_code(error_code) --> 544 raise error_class(parsed_response, operation_name) 545 else: ClientError: An error occurred (404) when calling the HeadObject operation: Not Found During handling of the above exception, another exception occurred: FileNotFoundError Traceback (most recent call last) <ipython-input-6-7628c3834aad> in <module>() ----> 1 fs.open('nyqpug/asdf.csv') /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in open(self, path, mode, block_size, acl, fill_cache, **kwargs) 289 " and manage bytes" % (mode[0] + 'b')) 290 return S3File(self, path, mode, block_size=block_size, acl=acl, --> 291 fill_cache=fill_cache, s3_additional_kwargs=kwargs) 292 293 def _lsdir(self, path, refresh=False): /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in __init__(self, s3, path, mode, block_size, acl, fill_cache, s3_additional_kwargs) 927 else: 928 try: --> 929 self.size = self.info()['Size'] 930 except (ClientError, ParamValidationError): 931 raise IOError("File not accessible", path) /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, **kwargs) 936 def info(self, **kwargs): 937 """ File information about this path """ --> 938 return self.s3.info(self.path, **kwargs) 939 940 def metadata(self, refresh=False, **kwargs): /Users/jreback/miniconda3/envs/pandas/lib/python3.6/site-packages/s3fs/core.py in info(self, path, refresh, **kwargs) 398 return out 399 except (ClientError, ParamValidationError): --> 400 raise FileNotFoundError(path) 401 402 _metadata_cache = {} FileNotFoundError: nyqpug/asdf.csv Metadata
Metadata
Assignees
Labels
IO CSVread_csv, to_csvread_csv, to_csvTestingpandas testing functions or related to the test suitepandas testing functions or related to the test suite