Resuming quest to open a massive text file

[Lyra]

Originally Posted by Aeolwind [You must be logged in to view images. Log in or Register.]

It will need to be sanitized obviously to protect the innocent.

This thread made me uncomfortable. What server were you on?

[getsome]

some troubleshooting questions.

Are you trying to open the file on this 2000 circa PC?

if so what operating system is it?

What file system is the hard drive where the txt file is stored? (You can right click the drive and select properties to look at the files system.)

From your earlier post it sounds like your old EQ machine is still functioning. In the interests of maintaining the integrity of your important log file you may want to create a copy before attempting anything. I recommend connecting the hard drive that contains the file to a USB hard drive adapter and make a copy of your log file to another media.

As for editors, I tested out a few.

I created a 4 GB test file using the following batch file.

echo "This is just a sample line appended  to create a big file. " > dummy.txt
for /L %%i in (1,1,26) do type dummy.txt >> dummy.txt

I currently use emEditor and it opened the file.

I also tried a free program that worked. http://www.hhdsoftware.com/free-hex-editor

[You must be logged in to view images. Log in or Register.]

[letsallkillandy]

There's also a high probability with a file size that large (and old) that it may have gone corrupt. Im frankly impressed you've accumulated 4gbs of TEXT. The largest log file i've seen was 1.5gbs and it barely opened.

[sabinrf24]

I recommend logparser 2.2 from Microsoft, of all sources. Pick whatever number increments make sense for you. I use this on 10-20GB logfiles regularly and I find it to be one of the best ways to handle them on Windows.

C:\Program Files (x86)\Log Parser 2.2>logparser -i:TEXTLINE "SELECT * FROM 'F:\P
rogram Files (x86)\Sony\EverQuest\Logs\eqlog_Character_project1 999.txt' WHERE Index
> 0 AND INDEX < 10000"

C:\Program Files (x86)\Log Parser 2.2>logparser -i:TEXTLINE "SELECT * FROM 'F:\P
rogram Files (x86)\Sony\EverQuest\Logs\eqlog_Character_project1 999.txt' WHERE Index
> 10000 AND INDEX < 20000"

logparser -h to see the export options, for CSV it would be:

C:\Program Files (x86)\Log Parser 2.2>logparser -i:TEXTLINE -o:CSV "SELECT * FROM 'F:\P
rogram Files (x86)\Sony\EverQuest\Logs\eqlog_Character_project1 999.txt' WHERE Index
> 10000 AND INDEX < 20000" > C:\folder\test.csv

[vulzol]

Originally Posted by letsallkillandy [You must be logged in to view images. Log in or Register.]

There's also a high probability with a file size that large (and old) that it may have gone corrupt. Im frankly impressed you've accumulated 4gbs of TEXT. The largest log file i've seen was 1.5gbs and it barely opened.

Hate to be a downer but this sounds probable.

[SCB]

I am so excited for you to post these logs to the forums as soon as you can get into the file.

[quido]

yeah honestly if you're trying to do this on an older computer, I'd just jimmy the HD out of it and stick it in a newer machine

[Thulack]

Good investment for 20 bucks is a IDE to USB device. I have one and love the ability to just take a old IDE HD out of a rig and in 5 seconds have a external HD out of it and can get whatever info off it at a reasonable pace.

[Pringles]

Unix / linux / osx: more / less / vi / cat will work just fine on 4gb file, I do it daily on logs much larger than this.

[sawin]

I also have been looking for a program to open very large files. I have considered using http://www.ultraedit.com/

There is a trial and it boasts the capacity to edit files over 4GB. I wish I could recommend it, but as I haven't yet tried it I'm unsure how well it would work for you.