[fixed] IDX file max size

You can tell us what you like or don't like in AlpineQuest, and what you would like to see in future releases.
MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

[fixed] IDX file max size

Post by MapWorm » Mon Dec 07, 2020 2:26 pm

Are there any limit set for the index file

Seems ours stuck on 2gig

Thx in advance

Psyberia-Support
Site Admin
Posts: 4801
Joined: Wed Apr 14, 2010 9:41 pm

Re: IDX file max size

Post by Psyberia-Support » Wed Dec 09, 2020 2:49 pm

Hi,

Yes, the index file (IDX) is limited to 2GB, which should handle around 100GB of stored data per map. You're the first to report reaching this limit.
This limit ensure a maximum compatibility (32bit linux without LFS only supports <2GB files on FAT32). This is also why the data is also split in 2GB files.

However, I guess it's quite secure to push the limit to 4GB for the index file.
Depending on your will to beta test this, you can install the last beta version (2.2.8.r6750) from the Play Store, I've made the changes.
But be careful, even if the changes are limited, I haven't been able to make test with a >2GB index (not sure if I'll ever be, my current larger map top to 10GB)...
Do you like AlpineQuest ? Leave a small comment on Google Play !

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Thu Dec 10, 2020 11:58 am

Wil do thx
My files topped #81 (162gb)

Since continue with it no "updates" was saved but still saved to next #82 chunk.

Only got one problem
Never made notes what was last saved correctly

Keep up the good work

I made some copy
Will see where Saved last
At least my placemarks are saved

Thx

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Thu Dec 10, 2020 12:20 pm

What I am running on now

Android 7
Offlinemaps 3.5c

256 gb sdcard formatted with format32 so phones limit of 128 gb can be bypassed

Working fine and fast
.....

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Thu Dec 10, 2020 12:28 pm

Do you have any android app which link to this site?

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Sat Dec 12, 2020 12:50 pm

Mercy mercy mercy has it that I did made a backup of all before reaching the idx file limit of 2gb. It was just 60mb shy of 2 gig

Reached #81

#81= 866 304 125 b (845.2 mb)
IDX= 2 088 079 933 b (1.9 gb)

Total calculated with totalcommander file properties

Past 2gb mark

#82 @ 880mb
IDX @ 2 146 642 413 b

Still saving ok in idx + dat files
When zooming in+out
Tiles show to be added
When going back to same level
IT'S SAVED
(not like before)

Well done fixing the problem

Will update as it the idx grows

....

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Sun Dec 13, 2020 5:34 am

IDX max capacity error
Error again past the 1.9gb point
Little larger than the normal chunk size
That's why I thought all is well at first

Today
2gb
2-147-531-813 bytes
Error
FAILED TO WRITE DATA: IOException (Failed to read the IDX file)

At first error in area download
Then ok in manual zoom in and out
Everything was saved ok but
After a while above error again while doing manual updates

Now continues above error

Kept step by step backups to reach and test this point faster

....

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size SOLUTION

Post by MapWorm » Sun Dec 13, 2020 6:32 am

Possible solution

Using Last ok saved idx
2-146-354-093 bytes

Because it's txt based entries the zip thought came to mind. If possible to make the idx a zip file. Then you can save 10x more tiles

Used totalcommander android for the test

1.9 gb file

Zipped idx files info versus speed
1.9 gb to....

108.7 mb - 480 sec - zip level 9
121.4 mb - 150 sec - zip level 6
131.5 mb - 80 sec - zip level 3
148.5 mb - 80 sec - zip level 1

Another thought of an idx for idx don't think it's possible

Keep up the good work

..,..

Psyberia-Support
Site Admin
Posts: 4801
Joined: Wed Apr 14, 2010 9:41 pm

Re: IDX file max size

Post by Psyberia-Support » Thu Dec 17, 2020 1:21 pm

Hi again,

Right now, no changes related to this problem are available in the current official versions, so you're still using a version of this application that is limited to a 2GB index file.

I've uploaded a BETA version of OfflineMaps that should work for index files up to 4GB (so you can store 2 times more data). The APK is available here: OfflineMaps_3.7.r6771.apk

Please note that:
- It's a beta version, with some new developments not extensively tested;
- The 4GB index file support has not been tested yet in a real use case;
- If you install this APK, it will replace the current installed version (you'll keep all your data, it will be like a regular app update). When a new version is available, it will be updated.
Do you like AlpineQuest ? Leave a small comment on Google Play !

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Mon Dec 21, 2020 5:50 pm

Thx a million
Will backup + test past 2 gb mark
Will let you know

MapWorm
Posts: 17
Joined: Thu Sep 10, 2020 10:36 am

Re: IDX file max size

Post by MapWorm » Mon Dec 28, 2020 6:24 pm

IDX size max out fix
Version 3.7.r6771

No errors so far
2.1 gb at last reached
Area download save correctly
Zoom in + out save correctly

When go back to saved areas no errors
Show as saved

Thx a lot

4 gig max will be far more than enough for my use

#81 @ 1.8 gb
idx @ 2.1 gb

Much appreciated

.....

Psyberia-Support
Site Admin
Posts: 4801
Joined: Wed Apr 14, 2010 9:41 pm

Re: [fixed] IDX file max size

Post by Psyberia-Support » Tue Jun 15, 2021 7:30 am

This changes are available from AlpineQuest 2.2.8 and OfflineMaps 3.6c.
Do you like AlpineQuest ? Leave a small comment on Google Play !

Post Reply