I have about 300,000 CSV files which I want to convert to shapefiles, but ArcGIS and QGIS can only convert one CSV at a time.
Are there any ways to batch convert my CSVfiles?
x y z1 z2 z3 z4 1 2 3 4 5 6 My csv files just have a few columns like above
Can any one help me?
If you really have ~300,000 csv files, I honestly hope that you don't want that many resultant shapefiles... For your sake, I hope that was a typo.
It is relatively simple to batch any tool in ArcGIS--you just right click on it and select 'batch'. I'd look into using Make XY Event Layer. As the layer is temporary, you'll want to save it. From the help:
The output point feature layer created by this tool is temporary and will not persist after the session ends. You can export this event layer to a feature class on disk using the Copy Features, Feature to Point, or Feature Class to Feature Class tool.
If you want just 1 shapefile, you need to concatenate the csv files. Something like How do I concatenate text files in Python? should get you started. You'll need to edit it some to not write the headers from each file.
If you can program in Python, get the osgeo bindings for ogr and gdal, then adapt something like this:
If you can program in R, get the sp and rgdal packages and then write a script to loop over them.
If you cannot program in anything, I suggest you learn!
If anyone is still ever looking at this and are interested in scripting instead of batching, you can use the glob module in python to search through your directory that contains the .csv's, and then use one of the arc tools (feature class to feature class, copy features, etc.) as mentioned above.
You can download the glob module for free, but I think it already exists in package.
I would do something like this,
from glob import glob directory = glob("DriveLetter:\FolderPath\\*.csv") -- grabs all .csv files and puts them into array called directory for files in directory: arcpy.MakeXYEventLayer_management(files, "xcoordField", "ycoordField", "outputName", "spatialReference", "zcoordFieldIfExists") arcpy.FeatureClassToFeatureClass_conversion("outputNameFromMakeXY", "YourDesiredOutputLocation", "NameofYourOutputFeatureClassorShapefile", "expressionIfEvenNeeded") If your .csv's all have the same naming for their fields this can be easy because you can hardwire them in, it is also best to have the spatial reference parameter as a GetParameterAsText(#), or just not give it a spatial reference and use the same script when finished and perform a define projection on your new shapefiles
you can also use python for the naming to only keep so many characters from each object in the directory array and using it for output names
This is a pretty clunky answer but gives you some hints to google for and try to make it work
You could also use dedicated ETL-tools (Extract, Transform and Load) like FME (licenssed software) or GeoKettle (open source) to do this. They will provide flexibility in processing your data to shapefiles. FME will allow you to "fan out" your data in several shapefiles depending on attributes.
Also if python programming is not your thing they will allow you to set up a job relatively effortlessly.
In ArcGIS you can do batch processing by right-clicking on the tool. See. http://resources.arcgis.com/en/help/main/10.1/index.html#//00210000000w000000.
