Re: Large files on Windows?

Hi Nedo,

When I try to open a file which is large 2,510,747,416 bytes (it is an 
aminoacidic sequence database in fasta format)


OK, looked up fasta and it should have short (< 200 byte) lines. Very long 
lines can cause performance problems. Extremely long lines may cause 
failures.

If you own the file or have permission to make it available, it may help to 
publish it, preferrably compressed.

, SciTE warns me that the file is larger than the 2,000,000,000 bytes limit 
set in the properties and I am asked if I still want to open it; when I 
click on "Yes" I wait for a while and then the SciTE window fills up with 
"NUL" texts and


 The most likely reason for NUL blobs appearing is that the file contains 
NUL bytes.

SciTE uses RAM to store the entire contents of files and also needs more 
memory to index lines, so may not work well on this file if your computer 
contains less than 4GB of RAM.
 

the vertical scroll bar is at the bottom.


That is unusual: SciTE normally shows the start of newly opened files.
 

If I move the scroll bar up above a certain point I start seeing the 
correct text I am expecting but then if I scroll back down, at a certain 
point, the "NUL" texts fills the SciTE window again.


That may mean that there are blocks of NUL bytes at the file end. 

I am aware that there are other options to deal with big files of this type 
but, despite this, is there anything that I can do to improve this with 
SciTE?


First determine if there is a problem: count the number of NUL bytes in the 
file with a script. Something like this piece of Python with fileName set 
to the path to your file.

fileName = "../bin/SciTE.exe"
import pathlib
bytes = pathlib.Path(fileName).read_bytes()
print(f"{fileName} contains {bytes.count(b'\0')} NULs in {len(bytes)} 
bytes")
 

I have searched this Google group with "file.size.large" and I read also 
the sentence from Neil Hodgson saying "An undocumented file.size.large 
property was added to SciTE that allowed loading files larger than 2GB. 
However, SciTE uses 32-bit integers for file positions so many features 
wouldn’t work past the 2GB point" but I am not able to say with certainty 
if there is something that can be done for my case.


The 64-bit version of SciTE has used 64-bit positions for over 3 years now 
and can load 2.5 GB files - I just loaded a 4GB file and it worked 
correctly.

Neil

-- 
You received this message because you are subscribed to the Google Groups "scite-interest" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/scite-interest/9b74e9f0-4e13-4482-acf2-9a164d3cba8fn%40googlegroups.com.