(Added 2016-01-05)
I created a new ticket at Github to all progress: https://github.com/gnosygnu/xowa/issues/36 . You can comment there or here, but I'll be adding all updates over there
(Added 2015-05-19)
So, I ran into some disheartening snags on English Wikipedia, and ended up postponing everything. Basically, I didn't realize that Android's implementation of FAT32 capped files at 2.1 GB (not 4.2). This pretty much renders useless the uploaded enwiki files at archive.org as they are 3 GB.
At this point, I'm going to focus on uploading an alpha for Simple Wikipedia for a rare Saturday release. I'm trying to get a private build of English Wikipedia working for myself, but I can use some testers for Simple Wikipedia. My primary concern is device compatibility, as I only have a 2.3 Froyo version, and ran into some problems with a brief encounter with KitKat. I'll post more instructions come Saturday.
(Added 2015-05-05)
My apologies for the lack of updates, but I've fallen badly behind schedule.
I'm at a point wherein I have a relatively stable beta of Simple Wikipedia. At this point, if you're interested in beta-testing Simple Wikipedia, or any other "small" wiki (< 1 GB dump size), I'll generate a test version for you sometimes by 5/15. Keep in mind Simple Wikipedia is about 2.2 GB, and the beta versions are disposable (you will probably need to download them again). So please only do this if you're interested in testing or helping out XOWA. English Wikipedia is still some time away, but hopefully if I get good feedback, I can get it out sooner.
I leave the rest of the earlier post intact below.
(Addeed 2014-10-27)
I posted the below to the blog page. I'm also posting below to get more visibility. Any feedback is welcome.
I thought it would be worthwhile to give a general overview, so that some expectations can be set.
XOWA will initially support Android 2.x (Eclair) and up. It won't support Alpha (1.0) Beta (1.1), Cupcake (1.5) or Donut (1.6). The limitation occurs because XOWA will be using the ActionBar Sherlock GUI library and it only supports 2.x+. If XOWA needs to support a pre 2.x version, I think it might be possible in a future release, but it won't be part of the initial release.
I'm planning three phases of release.
2014-12-01: Simple Wikipedia. This will include an XOWA prototype and a 2.2 GB database of text and images. I'm delaying this a month, because I want the database format to be reasonably final.
2015-01-05: English Wikipedia. This will include an XOWA beta and a 25 GB text-only database.
2015-02-03: Google play store. This will be the final version of the XOWA beta with a "full" version of English Wikipedia that will fit within a 64 GB microSD card. In addition, I'll have a German Wikipedia available as well, with other versions to follow soon afterwards.
If you have an Android device, I'd really appreciate it if you do some beta testing for the first two phases. In particular, I'd like feedback on how XOWA works on your device in terms of the following:
Speed: Does XOWA retrieve pages fairly quickly? I have a Samsung Galaxy 2.2 phone, so I think I have the lower end of the spectrum covered, but I'd like to have feedback on as many different devices as possible.
Display: Does XOWA look fine at your screen resolution? How about portrait vs landscape?
Other issues: Android and its multiple frameworks can be challenging, so I'm sure there will be issues that will arise on certain devices.
Finally, XOWA will be using a new database html dump format. Android is too slow to compile wikitext, especially when you throw in Luaj and Jtidy (some pages would take 2+ minutes on my phone). The advantage of the new format is that it will be much faster (less than 2 seconds for any page). Unfortunately, the disadvantages are several:
New databases will have to be built. I'm working on a script file to compile the existing wikitext databases to html databases, but in the beginning, I'll probably be providing these html databases separately.
Wikitext won't be available. There will be an option to combine a wikitext database and an html database, but the resulting file will probably be 50 GB (without even the images).
I have to rewrite parts of XOWA to handle an "html-only" version. I've been doing this for the past few months, but will need to do more over the next two.
Last edit: gnosygnu 2016-01-05
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
Anonymous
-
2014-11-20
Can I make a plea for an early release of English Wikisource Android?
It would be fantastic to have so much literature on a tablet.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Can I make a plea for an early release of English Wikisource Android?
Hi! This could be possible. I'm trying to get images done, but English Wikisource doesn't rely as much on images. I'll kick off an English Wikisource dump either next week or the following and see how that looks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
Anonymous
-
2014-11-22
That's great.
Many thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hmmm.... I made one mistake in saying that simple.wikipedia.org-file-user.xowa should be at https://archive.org/details/Xowa_simplewiki_latest . I don't have access now, but it probably is not there. However, simple.wikipedia.org-text.xowa and simple.wikipedia.org-file.xowa should be there. Are you not seeing either?
Is this one needed?
simple.wikipedia.org-html.xowa
Yes, you will also need the simple.wikipedia.org-html.xowa database.
Is this one missing?
simple.wikipedia.org-file-user.xowa
Yes, it is missing, but it isn't necessary.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It can be omitted. But at this point, I'd wait until this Sunday for the new alpha. It is much improved over the current (I've been making fixes for it and English Wikipedia throughout the vacation)
Thanks.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
So, I ran into some disheartening snags on English Wikipedia, and ended up postponing everything. Basically, I didn't realize that Android's implementation of FAT32 capped files at 2.1 GB (not 4.2). This pretty much renders useless the uploaded enwiki files at archive.org as they are 3 GB.
I do mot hear about this before. Maybe this is interesting for you (not a solution):
Thanks for the links. LFS is still an option. However I wanted to upload files that were as simple to use as possible. 3 GB files means a little more work for the user, and it's something I'd like to avoid.
I'm planning to upload a new set of 2 GB files, but will support the 3 GB and 2 GB set going forward.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
No, unfortunately, this is an issue on my side. The alpha doesn't support KitKat which is probably what's causing it to to fail. I fixed this on my side, but won't have another alpha ready until 6-15. The good news is that I've been working with an English Wikipedia on a KitKat phone, and it works fairly well. I'm working to have an English Wikipedia set uploaded by 6-30.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ok, next try with the new version.
Uninstall old version, install new version.
Import Simple Wikipedia
XOWA opens up to its own file browser (done, ok)
Navigate to the folder for simple.wikipedia.org (done, ok)
Check the box for simple.wikipedia.org-text.xowa (done, ok)
Click the "import" link in the upper-right hand corner. You may have to scroll to the right to find the link. (done, ok)
Click the "simple.wikipedia.org" link. It will take you to the Earth page. (done, error)
Thanks for trying it out, as well as the screenshots!
Are you sure you downloaded the latest simple.wikipedia.org-text.xowa from https://sourceforge.net/projects/xowa/files/support/alphas/? I downloaded it now, and it should have the css data. I also ran these on two separate Android devices on Sunday night and it worked then.
You can confirm if your .xowa file has the css data by doing the following:
Open simple.wikipedia.org-text.xowa with sqlite3. If you're on windows, you can use the sqlite3 shell in "/xowa/bin/windows/sqlite/".
Run "SELECT * FROM css_core;". You should get something like the following:
(Added 2016-01-05)
I created a new ticket at Github to all progress: https://github.com/gnosygnu/xowa/issues/36 . You can comment there or here, but I'll be adding all updates over there
(Added 2015-05-24)
I uploaded the Android alpha tonight. The files are here: https://sourceforge.net/projects/xowa/files/support/alphas/. Instructions are in [[Help:Android/Alpha]] in the latest version.
(Added 2015-05-19)
So, I ran into some disheartening snags on English Wikipedia, and ended up postponing everything. Basically, I didn't realize that Android's implementation of FAT32 capped files at 2.1 GB (not 4.2). This pretty much renders useless the uploaded enwiki files at archive.org as they are 3 GB.
At this point, I'm going to focus on uploading an alpha for Simple Wikipedia for a rare Saturday release. I'm trying to get a private build of English Wikipedia working for myself, but I can use some testers for Simple Wikipedia. My primary concern is device compatibility, as I only have a 2.3 Froyo version, and ran into some problems with a brief encounter with KitKat. I'll post more instructions come Saturday.
(Added 2015-05-05)
My apologies for the lack of updates, but I've fallen badly behind schedule.
I'm at a point wherein I have a relatively stable beta of Simple Wikipedia. At this point, if you're interested in beta-testing Simple Wikipedia, or any other "small" wiki (< 1 GB dump size), I'll generate a test version for you sometimes by 5/15. Keep in mind Simple Wikipedia is about 2.2 GB, and the beta versions are disposable (you will probably need to download them again). So please only do this if you're interested in testing or helping out XOWA. English Wikipedia is still some time away, but hopefully if I get good feedback, I can get it out sooner.
I leave the rest of the earlier post intact below.
(Addeed 2014-10-27)
I posted the below to the blog page. I'm also posting below to get more visibility. Any feedback is welcome.
I thought it would be worthwhile to give a general overview, so that some expectations can be set.
XOWA will initially support Android 2.x (Eclair) and up. It won't support Alpha (1.0) Beta (1.1), Cupcake (1.5) or Donut (1.6). The limitation occurs because XOWA will be using the ActionBar Sherlock GUI library and it only supports 2.x+. If XOWA needs to support a pre 2.x version, I think it might be possible in a future release, but it won't be part of the initial release.
I'm planning three phases of release.
If you have an Android device, I'd really appreciate it if you do some beta testing for the first two phases. In particular, I'd like feedback on how XOWA works on your device in terms of the following:
Finally, XOWA will be using a new database html dump format. Android is too slow to compile wikitext, especially when you throw in Luaj and Jtidy (some pages would take 2+ minutes on my phone). The advantage of the new format is that it will be much faster (less than 2 seconds for any page). Unfortunately, the disadvantages are several:
Last edit: gnosygnu 2016-01-05
Can I make a plea for an early release of English Wikisource Android?
It would be fantastic to have so much literature on a tablet.
Hi! This could be possible. I'm trying to get images done, but English Wikisource doesn't rely as much on images. I'll kick off an English Wikisource dump either next week or the following and see how that looks.
That's great.
Many thanks.
No problem, but save the thanks for when I actually produce the dump!
I uploaded the Android alpha tonight. The files are here: https://sourceforge.net/projects/xowa/files/support/alphas/. Instructions are in [[Help:Android/Alpha]] in the latest version.
Your advice in [[Help:Android/Alpha]]
I found this two files in the archive:
simple.wikipedia.org-html.xowa
simple.wikipedia.org-text.xowa
This one simple.wikipedia.org-file.xowa is here:
Is this one needed?
simple.wikipedia.org-html.xowa
Is this one missing?
simple.wikipedia.org-file-user.xowa
Last edit: Anselm D 2015-06-03
Hmmm.... I made one mistake in saying that simple.wikipedia.org-file-user.xowa should be at https://archive.org/details/Xowa_simplewiki_latest . I don't have access now, but it probably is not there. However, simple.wikipedia.org-text.xowa and simple.wikipedia.org-file.xowa should be there. Are you not seeing either?
Yes, you will also need the simple.wikipedia.org-html.xowa database.
Yes, it is missing, but it isn't necessary.
Screenshot of the archive:
Cool. Thanks for the screenshot. So...
Let me know if I'm misunderstanding anything. Thanks.
Do i need the simple.wikipedia.org-file-user.xowa to start xowa? Or can it be ommited?
It can be omitted. But at this point, I'd wait until this Sunday for the new alpha. It is much improved over the current (I've been making fixes for it and English Wikipedia throughout the vacation)
Thanks.
Whats about the files
Xowa_simplewiki_latest_meta.sqlite
Xowa_simplewiki_latest_meta.xml
Do i need them?
Nope. These files are generated automatically by archive.org for their own purposes. XOWA doesn't use them.
I do mot hear about this before. Maybe this is interesting for you (not a solution):
File Allocation Table - Wikipedia, the free encyclopedia
http://en.wikipedia.org/wiki/File_Allocation_Table#FAT32
"...
Max. file size 2,147,483,647 bytes (2 GB - 1) (without LFS)
4,294,967,295 bytes (4 GB - 1) (with LFS)
limited by volume size only (with FAT16+[34])
..."
Comparison of file systems - Wikipedia, the free encyclopedia
http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
"...
2 [[GB (4 [[GB with LFS
..."
Large file support - Wikipedia, the free encyclopedia
http://en.wikipedia.org/wiki/Large_file_support
FAT32 file size limited to 2GB–Google Groups
https://groups.google.com/forum/#!topic/android-porting/alF-avcQNfI
Thanks for the links. LFS is still an option. However I wanted to upload files that were as simple to use as possible. 3 GB files means a little more work for the user, and it's something I'd like to avoid.
I'm planning to upload a new set of 2 GB files, but will support the 3 GB and 2 GB set going forward.
FYI:To get a feeling for it, I installed it at Andy Android Emulator.
Andy the Android Emulator
http://www.andyroid.net/
So the result is not meaningful.
Import Simple Wikipedia
I am not able to navigate to the sdcard (clicking with mouse one time at sdcard in XowaFileBrowser)
Do you think this is a weakness in the emulator? Or a problem with the access rights?
Last edit: Anselm D 2015-06-04
No, unfortunately, this is an issue on my side. The alpha doesn't support KitKat which is probably what's causing it to to fail. I fixed this on my side, but won't have another alpha ready until 6-15. The good news is that I've been working with an English Wikipedia on a KitKat phone, and it works fairly well. I'm working to have an English Wikipedia set uploaded by 6-30.
Ok, next try with the new version.
Uninstall old version, install new version.
Import Simple Wikipedia
Click the "simple.wikipedia.org" link. It will take you to the Earth page. (done, error)
Last edit: Anselm D 2015-06-18
For the Bluestacks Android emulator i am getting another stack trace.
Last edit: Anselm D 2015-06-18
With Genymotion
I chose a Samsung Galaxy Note 3 - 4.4.4 - API 19
I am getting the same error as with Bluestacks
Do you think this are difficulties of the emulator or I did something wrong?
Last edit: Anselm D 2015-06-18
Thanks for trying it out, as well as the screenshots!
Are you sure you downloaded the latest simple.wikipedia.org-text.xowa from https://sourceforge.net/projects/xowa/files/support/alphas/? I downloaded it now, and it should have the css data. I also ran these on two separate Android devices on Sunday night and it worked then.
You can confirm if your .xowa file has the css data by doing the following:
If you do have these rows, but it still doesn't work on the emulator, I'll investigate more.
Thanks again!
You are right, i did not recognize it.
We struck gold! :-)
Now it started with Andy (Android Version 4.2.2; GT-P5200)
I did some small tests and it works fine!
You did a great job!
The same with blue stack
(Android Version 4.4.2; Model GT-N7100)
Genymotion:Samsung Galaxy Note 2 - 4.3 - API 18 - 720x1280