Page 1 of 1
[fixed] IDX file max size
Posted: Mon Dec 07, 2020 2:26 pm
by MapWorm
Are there any limit set for the index file
Seems ours stuck on 2gig
Thx in advance
Re: IDX file max size
Posted: Wed Dec 09, 2020 2:49 pm
by Psyberia-Support
Hi,
Yes, the index file (IDX) is limited to 2GB, which should handle around 100GB of stored data per map. You're the first to report reaching this limit.
This limit ensure a maximum compatibility (32bit linux without LFS only supports <2GB files on FAT32). This is also why the data is also split in 2GB files.
However, I guess it's quite secure to push the limit to 4GB for the index file.
Depending on your will to beta test this, you can install the last beta version (2.2.8.r6750) from the Play Store, I've made the changes.
But be careful, even if the changes are limited, I haven't been able to make test with a >2GB index (not sure if I'll ever be, my current larger map top to 10GB)...
Re: IDX file max size
Posted: Thu Dec 10, 2020 11:58 am
by MapWorm
Wil do thx
My files topped #81 (162gb)
Since continue with it no "updates" was saved but still saved to next #82 chunk.
Only got one problem
Never made notes what was last saved correctly
Keep up the good work
I made some copy
Will see where Saved last
At least my placemarks are saved
Thx
Re: IDX file max size
Posted: Thu Dec 10, 2020 12:20 pm
by MapWorm
What I am running on now
Android 7
Offlinemaps 3.5c
256 gb sdcard formatted with format32 so phones limit of 128 gb can be bypassed
Working fine and fast
.....
Re: IDX file max size
Posted: Thu Dec 10, 2020 12:28 pm
by MapWorm
Do you have any android app which link to this site?
Re: IDX file max size
Posted: Sat Dec 12, 2020 12:50 pm
by MapWorm
Mercy mercy mercy has it that I did made a backup of all before reaching the idx file limit of 2gb. It was just 60mb shy of 2 gig
Reached #81
#81= 866 304 125 b (845.2 mb)
IDX= 2 088 079 933 b (1.9 gb)
Total calculated with totalcommander file properties
Past 2gb mark
#82 @ 880mb
IDX @ 2 146 642 413 b
Still saving ok in idx + dat files
When zooming in+out
Tiles show to be added
When going back to same level
IT'S SAVED
(not like before)
Well done fixing the problem
Will update as it the idx grows
....
Re: IDX file max size
Posted: Sun Dec 13, 2020 5:34 am
by MapWorm
IDX max capacity error
Error again past the 1.9gb point
Little larger than the normal chunk size
That's why I thought all is well at first
Today
2gb
2-147-531-813 bytes
Error
FAILED TO WRITE DATA: IOException (Failed to read the IDX file)
At first error in area download
Then ok in manual zoom in and out
Everything was saved ok but
After a while above error again while doing manual updates
Now continues above error
Kept step by step backups to reach and test this point faster
....
Re: IDX file max size SOLUTION
Posted: Sun Dec 13, 2020 6:32 am
by MapWorm
Possible solution
Using Last ok saved idx
2-146-354-093 bytes
Because it's txt based entries the zip thought came to mind. If possible to make the idx a zip file. Then you can save 10x more tiles
Used totalcommander android for the test
1.9 gb file
Zipped idx files info versus speed
1.9 gb to....
108.7 mb - 480 sec - zip level 9
121.4 mb - 150 sec - zip level 6
131.5 mb - 80 sec - zip level 3
148.5 mb - 80 sec - zip level 1
Another thought of an idx for idx don't think it's possible
Keep up the good work
..,..
Re: IDX file max size
Posted: Thu Dec 17, 2020 1:21 pm
by Psyberia-Support
Hi again,
Right now, no changes related to this problem are available in the current official versions, so you're still using a version of this application that is limited to a 2GB index file.
I've uploaded a BETA version of OfflineMaps that should work for index files up to 4GB (so you can store 2 times more data). The APK is available here:
OfflineMaps_3.7.r6771.apk
Please note that:
- It's a beta version, with some new developments not extensively tested;
- The 4GB index file support has not been tested yet in a real use case;
- If you install this APK, it will replace the current installed version (you'll keep all your data, it will be like a regular app update). When a new version is available, it will be updated.
Re: IDX file max size
Posted: Mon Dec 21, 2020 5:50 pm
by MapWorm
Thx a million
Will backup + test past 2 gb mark
Will let you know
Re: IDX file max size
Posted: Mon Dec 28, 2020 6:24 pm
by MapWorm
IDX size max out fix
Version 3.7.r6771
No errors so far
2.1 gb at last reached
Area download save correctly
Zoom in + out save correctly
When go back to saved areas no errors
Show as saved
Thx a lot
4 gig max will be far more than enough for my use
#81 @ 1.8 gb
idx @ 2.1 gb
Much appreciated
.....
Re: [fixed] IDX file max size
Posted: Tue Jun 15, 2021 7:30 am
by Psyberia-Support
This changes are available from AlpineQuest 2.2.8 and OfflineMaps 3.6c.
Re: [fixed] IDX file max size
Posted: Tue Mar 07, 2023 12:40 pm
by MapWorm
Just a feedback
Idx now at 2.1gb
Dat on 1.9 gb
And growing strong
Thx team
Keep up the good work
MapWorm