Page 1 of 1

[fixed] IDX file max size

Posted: Mon Dec 07, 2020 2:26 pm
by MapWorm
Are there any limit set for the index file

Seems ours stuck on 2gig

Thx in advance

Re: IDX file max size

Posted: Wed Dec 09, 2020 2:49 pm
by Psyberia-Support
Hi,

Yes, the index file (IDX) is limited to 2GB, which should handle around 100GB of stored data per map. You're the first to report reaching this limit.
This limit ensure a maximum compatibility (32bit linux without LFS only supports <2GB files on FAT32). This is also why the data is also split in 2GB files.

However, I guess it's quite secure to push the limit to 4GB for the index file.
Depending on your will to beta test this, you can install the last beta version (2.2.8.r6750) from the Play Store, I've made the changes.
But be careful, even if the changes are limited, I haven't been able to make test with a >2GB index (not sure if I'll ever be, my current larger map top to 10GB)...

Re: IDX file max size

Posted: Thu Dec 10, 2020 11:58 am
by MapWorm
Wil do thx
My files topped #81 (162gb)

Since continue with it no "updates" was saved but still saved to next #82 chunk.

Only got one problem
Never made notes what was last saved correctly

Keep up the good work

I made some copy
Will see where Saved last
At least my placemarks are saved

Thx

Re: IDX file max size

Posted: Thu Dec 10, 2020 12:20 pm
by MapWorm
What I am running on now

Android 7
Offlinemaps 3.5c

256 gb sdcard formatted with format32 so phones limit of 128 gb can be bypassed

Working fine and fast
.....

Re: IDX file max size

Posted: Thu Dec 10, 2020 12:28 pm
by MapWorm
Do you have any android app which link to this site?

Re: IDX file max size

Posted: Sat Dec 12, 2020 12:50 pm
by MapWorm
Mercy mercy mercy has it that I did made a backup of all before reaching the idx file limit of 2gb. It was just 60mb shy of 2 gig

Reached #81

#81= 866 304 125 b (845.2 mb)
IDX= 2 088 079 933 b (1.9 gb)

Total calculated with totalcommander file properties

Past 2gb mark

#82 @ 880mb
IDX @ 2 146 642 413 b

Still saving ok in idx + dat files
When zooming in+out
Tiles show to be added
When going back to same level
IT'S SAVED
(not like before)

Well done fixing the problem

Will update as it the idx grows

....

Re: IDX file max size

Posted: Sun Dec 13, 2020 5:34 am
by MapWorm
IDX max capacity error
Error again past the 1.9gb point
Little larger than the normal chunk size
That's why I thought all is well at first

Today
2gb
2-147-531-813 bytes
Error
FAILED TO WRITE DATA: IOException (Failed to read the IDX file)

At first error in area download
Then ok in manual zoom in and out
Everything was saved ok but
After a while above error again while doing manual updates

Now continues above error

Kept step by step backups to reach and test this point faster

....

Re: IDX file max size SOLUTION

Posted: Sun Dec 13, 2020 6:32 am
by MapWorm
Possible solution

Using Last ok saved idx
2-146-354-093 bytes

Because it's txt based entries the zip thought came to mind. If possible to make the idx a zip file. Then you can save 10x more tiles

Used totalcommander android for the test

1.9 gb file

Zipped idx files info versus speed
1.9 gb to....

108.7 mb - 480 sec - zip level 9
121.4 mb - 150 sec - zip level 6
131.5 mb - 80 sec - zip level 3
148.5 mb - 80 sec - zip level 1

Another thought of an idx for idx don't think it's possible

Keep up the good work

..,..

Re: IDX file max size

Posted: Thu Dec 17, 2020 1:21 pm
by Psyberia-Support
Hi again,

Right now, no changes related to this problem are available in the current official versions, so you're still using a version of this application that is limited to a 2GB index file.

I've uploaded a BETA version of OfflineMaps that should work for index files up to 4GB (so you can store 2 times more data). The APK is available here: OfflineMaps_3.7.r6771.apk

Please note that:
- It's a beta version, with some new developments not extensively tested;
- The 4GB index file support has not been tested yet in a real use case;
- If you install this APK, it will replace the current installed version (you'll keep all your data, it will be like a regular app update). When a new version is available, it will be updated.

Re: IDX file max size

Posted: Mon Dec 21, 2020 5:50 pm
by MapWorm
Thx a million
Will backup + test past 2 gb mark
Will let you know

Re: IDX file max size

Posted: Mon Dec 28, 2020 6:24 pm
by MapWorm
IDX size max out fix
Version 3.7.r6771

No errors so far
2.1 gb at last reached
Area download save correctly
Zoom in + out save correctly

When go back to saved areas no errors
Show as saved

Thx a lot

4 gig max will be far more than enough for my use

#81 @ 1.8 gb
idx @ 2.1 gb

Much appreciated

.....

Re: [fixed] IDX file max size

Posted: Tue Jun 15, 2021 7:30 am
by Psyberia-Support
This changes are available from AlpineQuest 2.2.8 and OfflineMaps 3.6c.

Re: [fixed] IDX file max size

Posted: Tue Mar 07, 2023 12:40 pm
by MapWorm
Just a feedback

Idx now at 2.1gb
Dat on 1.9 gb

And growing strong

Thx team

Keep up the good work

MapWorm