Using chdir() to read all csv files in a directory [duplicate]

Question

Here is an example of my directory:

Auto ├── Test │ └── Test.csv ├── Test2 │ └── Test2.csv └── new.csv

I am trying to read all the csv files in the directory.

Here is my code:

import glob, os os.chdir('C:\\Users\\Me\\Desktop\\Auto') for file in glob.iglob('*.csv'): print(file)

This only prints “new.csv” and not “Test.csv” or “Test2.csv”.

I was thinking maybe I could try the directory name as the following:

os.chdir('C:\\Users\\Me\\Desktop\\Auto\\{}')

But, this gives me a FileNotFoundFoundError as the system cannot find the directory specified.

There's a canonical duplicate for this regardless of what the actual directory structure is, but I think something's wrong with the diagram. Is Test2 supposed to be inside Test, or directly within Auto? Why are there two separate lines pointing at new.csv? — Karl Knechtel
– Karl Knechtel, Commented Jan 9, 2023 at 22:41
@KarlKnechtel I probably did mess up the diagram. I have a folder named “Auto”. Inside the “Auto” folder I have a csv file named “new” and 2 more folders named “Test” and “Test2”. Inside the “Test” folder there is a csv file named “Test” and inside the “Test2” folder there is a csv file named “Test2”. I want to print all 3 csv files. — GooseCake
– GooseCake, Commented Jan 9, 2023 at 22:47
I fixed the diagram to match that description. Please see the linked duplicate for an answer to your question, and meta.stackoverflow.com/questions/420280 for help with making such diagrams. — Karl Knechtel
– Karl Knechtel, Commented Jan 9, 2023 at 22:50
Using chdir would really not be a reasonable approach IMO. As noted in the duplicate or the answer below, you don't have to do anything because the glob module can handle this, but even if it couldn't, you should instead use an absolute path and construct it before passing it to glob, don't actually change the working directory — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Jan 9, 2023 at 23:12

phibel · Accepted Answer · 2023-01-09 22:36:15Z

1

glob iterates by default only through the root dir, no subdirectories

import glob, os for file in glob.iglob(r'C:\Users\Me\Desktop\Auto\**\*.csv', recursive=True): print(file)

answered Jan 9, 2023 at 22:36

4012 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

I have changed the code to add recursive=True and it still only shows “new.csv”

Have you also modified your path? See my example, you need **\\*.csv