I have been using pylzma for a little bit, but I have to be able to create files compatible with the 7zip windows application. The caveat is that some of my files are really large (3 to 4gb created by a third party software in a proprietary binary format).
I went over and over here and on the instructions here: https://github.com/fancycode/pylzma/blob/master/doc/USAGE.md
I am able to create compatible files with the following code:
def Compacts(folder,f): os.chdir(folder) fsize=os.stat(f).st_size t=time.clock() i = open(f, 'rb') o = open(f+'.7z', 'wb') i.seek(0) s = pylzma.compressfile(i) result = s.read(5) result += struct.pack('<Q', fsize) s=result+s.read() o.write(s) o.flush() o.close() i.close() os.remove(f) The smaller files (up to 2Gb) compress well with this code and are compatible with 7Zip, but the larger files just crash python after some time.
According to the user guide, to compact large files one should use streaming, but then the resulting file is not compatible with 7zip, as in the snippet bellow.
def Compacts(folder,f): os.chdir(folder) fsize=os.stat(f).st_size t=time.clock() i = open(f, 'rb') o = open(f+'.7z', 'wb') i.seek(0) s = pylzma.compressfile(i) while True: tmp = s.read(1) if not tmp: break o.write(tmp) o.flush() o.close() i.close() os.remove(f) Any ideas on how can I incorporate the streaming technique present in pylzma while keeping the 7zip compatibility?
