Welcome to Wikilivres! The site has recently moved servers. Please re-register your account.

Project:Community Portal/en

Free texts and images.
Jump to: navigation, search

Add a topic

Forums: English · French · Polish · Russian · Spanish

Visit the Community Portal Archives


Bibliowiki hosting

We should have a community-wide discussion about our long-term hosting plan. I am willing to help financially and logistically as I can but note that I am located in the United States and I am an American citizen. First off: do we have any Canadian resident editors here who are interested in this responsibility? Koavf (talk) 07:45, 12 August 2018 (UTC)

I'm not sure why one needs to be a Canadian citizen to host content in Canada. I'm certainly not (Canadian) and it is the easiest thing to deal with VPS cloud providers (there are many operating in Canada). Our current host is SSDNodes.com and we have a great deal with them ($9.99/mo), but they full and cannot provide additional resources. Amazon AWS is offered in Montreal, both EC2 and now LightSail. All of these do require technical knowlege in setting up and securing a server (which is not trivial). Note that there is one Mediawiki-specified host in Canada which offers MediaWiki hosting, though it is not clear the extent of support provided. Should be worth contacting them: https://www.mezzohost.com/webhosting/mediawiki/ --Sysadmin (talk) 04:28, 13 August 2018 (UTC)
Note: Koavf's post was prompted by a message I sent to all donors of Biblio.wiki, as follows:
Hi guys,
On Nov 1st, I'll be shutting down the virtual private server that has https://biblio.wiki hosted on it. After that, I will not be running any servers that will be able to host Biblio.wiki. This means I'm getting out of the Mediawiki hosting business, and that means the Biblio.wiki website will need to find another virtual home (or take over the current account at SSDNodes for hosting) at that point, and also someone to do the maintenance on it.
Unfortunately I don't have any recommendations to make regarding who/where to do the hosting and maintenance. People who have read the technical and policy discussion pages are already familiar with my consideration of the viability of the platform (not good). Unfortunately, those few who are active on the site generally use the functionality that makes it uniquely Mediawiki, especially running bots and the djvu-support.
Right now the site uses 37gb out of a 40gb limit, which is not sustainable. File growth is a major issue on the site as it is currently configured. I've tried to think of various ways around it, but I find I simply don't have the time to try and deal with this properly, and provide the support the project needs to remain healthy. My lack of time is partially due to the fact that at home we have a growing 8 month old child, in addition to our 2 year, 9 month old child.
I started helping out this project about 18 months ago and we did get some level of uptime going for a system which had shut down and all that was available was data backups. It is now time for someone else to take on the opportunity to serve the Biblio.wiki community in this way. I appreciate all the trust and patience you have offered me over the previous 18 months to help out in this technical capacity.
November 1st is a hard deadline, so getting to work and doing a transition/migration earlier than that is wise.
Please use the bibliowiki site to discuss this issue and it will be a community decision on whom will get the technical keys to the kingdom, with all who contributed finanically so far, to have a vote.
- wiith regards, --Sysadmin (talk) 04:32, 13 August 2018 (UTC)
To say the truth I am puzzed what to say... My hands slipped... I am not familiar with servers and hosting. I am pretty good with editing only. I have not idea what to do at the moment. Maybe it would be a good idea to move on a free wiki farm like Miraheze? But it is an American firm... Or transfer our content to wikilivres.ru, as I proposed in previous time. Electron   10:19, 18 August 2018 (UTC)

I am puzzled by the claim that "those few who are active on the site generally use the functionality that makes it uniquely Mediawiki, especially running bots and the djvu-support." I am one of the most active users and have never used either function. All I need is the ability to upload text, ideally with certain formatting capabilities such as bold and italic. Thus I would be quite content not to continue with wiki software. I would have some problems with trying to use wikilivres.ru, as the interface is in Russian. Who owns and operates this site? Does Russian copyright law allow for a 50 year cut-off? --Poetlister

For me Mediawiki is essential. I have to say it: no Mediawiki, no my work... I have no time to learn new type of wiki and change more that 30.000 pages to fit them new type of formatting. Wikilivres.ru is run by Dmitrismirnov. It was not against about our moveing to the site. Main interface every mediawiki can be changed to many languages in yours preferences when you are logged in. I suppose Russian are not as strike to copyright rules as others, so we would abide our previous rules... But if you have better proposition I am not against... Electron   11:49, 19 August 2018 (UTC)
Russia has a Life+70 copyright regime, which would make most content illegal if hosted on wikilivres.ru, unfortunately. If they moved hosting to Belarus that would not be an issue (and Belarus has many hosting options), since that is one of the last remaining countries in Europe with a Life+50 copyright regime. Mezzohost as mentioned above is a Canada-based Mediawiki-specific hosting company. I would think finding out their options would be a good step forward. P.S., note that the Miraheze.org site is hosted in the Netherlands, a Life+70 regime. --Sysadmin (talk) 03:43, 20 August 2018 (UTC)
Also note that Miraheze.org does not have the Proofread extention, which is basically the main impediment to migrating to a different wiki platform (dokuwiki), see: https://meta.miraheze.org/wiki/Extensions --Sysadmin (talk) 03:52, 20 August 2018 (UTC)
Maybe they install it on our request? If it is important for us... See: If you would like to enable other extensions on your wiki that you don't see on this list, please make a task on Phabricator. -> ManageWiki#Extensions Electron   11:21, 20 August 2018 (UTC)
Well, I am not against Belarus... But, well... it is not democratic country and the country of law. Less than Russia, to say the truth... But if there is no better alternative... Electron   11:13, 20 August 2018 (UTC)

That site is not hosted in Belarus. Sysadmin was just saying that if they moved to Belarus we would be OK under the 50-year rule. I am quite happy with Mezzohost.--Poetlister 19:54, 21 August 2018 (UTC)

Looks not bad. They offer price starting at $29/year and a 15% discount for non-profit organizations and community groups. But I haven't found their regular prices... Electron   10:25, 23 August 2018 (UTC)

I've called and written them several times with no response. Koavf (talk) 04:02, 27 August 2018 (UTC)

I made some enquiries via the Wikipediocracy.com website and had the following response from User Dysklyver there. If it is helpful, I can put you in contact. (He tried to create an account here but could not; I told him to e-mail info@biblio.wiki. He also notes that Special:Version is missing. In fact, it is ther ebut can't be seen when logged out.)
I am quite the expert in setting up MediaWiki now, I have done 11 sites and multiple complex installs such as wikibase etc, if you guys need help moving server I am happy to assist. AWS and Google Cloud offer Canadian hosting, the system admins do not need to be Canadian. My recommendation would be to use one of these.
I would not recommend a Mediawiki specific server farm such as Mezzohost - you could use it, but I strongly warn you not to. Without root access you will have problems with extensions and file hosting. Although I joke about maybe hosting my blog in Russia, this is far from ideal and should be avoided, as really should Belarus. Miraheze is not an option currently due to extensions, it is entirely US based and not an option due to copyright. Google Cloud allows for easy drive expansion, AWS is not so good at this, since their pricing and services are comparable I would suggest Google Cloud, this is where I host World Wiki.
Your existing package that costs $9.99 a month is very cheap compared to Google's quoted $28 a month. It could be beneficial to offload image storing to another server and keep your current package.
In conclusion I think the best idea would be to host all the images, divj's etc in a flat file system separate to the server. This is reasonably easy to setup with something like Extension:CloudStorage.
  • I have registered here now, my current thoughts are that there are four clear options:
  1. Remain using the current package and limit the number of files to avoid running out of storage.
  2. Move the wiki to a larger server which would be more expensive.
  3. Move the wiki to a split format where the images are in an expandable cloud storage system and not on the server.
  4. Move the wiki onto someone else's infrastructure.
If that helps. Dysklyver (talk) 16:54, 3 September 2018 (UTC)
Note that AWS just cut their Lightsail VPS prices in half, and they do have Canada locations. This I think is the best approach, except for just keeping the site where it is. While we do have 4 processors and 8gb of ram, I've never seen it go above 4gb of ram so my guess is the other 4gb isn't really accessible. Lightsail is a much better deal than EC2 because it includes several DNS zones, a whole lot of throughput/bandwidth, and storage space. As long as Bibliowiki has the technical skills to keep their own installation up and running, then staying on SSDnodes or moving to Amazon Lightsail seems to me the most viable VPS hosting.
Poetlister's and Dysklyver's thoughts on dealing with images using an external location (as one option) mirror my own, though some clever compression might make that less critical (I've not yet addressed the compression issue, running compression tools against the file system, though it does have promise). It is reassuring that some of the editors have the technical skills needed to keep this project going, which I was not aware of up until now. --Sysadmin (talk) 15:09, 10 September 2018 (UTC)
With the current host, it appears to be possible to upgrade the current server from 40gb to 80gb. However, the new plan is $167.99 annually (payable upon upgrade). This would basically solve the biggest issue which strikes every few months when the server runs out of space. It is also not much more than we pay currently ($14/mo vs. $10/mo). We have maybe $20 or so in the current donations (I have to do the accounting, there were some recent donations), but I think it might be possible to find an external sponsor who could pay effectively $120 for the year for a few links/name recognition as a sponsor. Let me know the thoughts on this. Editor/admin donations barely keep up with current expenses. --Sysadmin (talk) 05:14, 23 September 2018 (UTC)
There are complications with upgrading the host. The virtual server technology for their current offering is different from what we are currently running (OpenVZ vs. KVM), therefore there is no painless upgrade path. It would require backup and restore of files, but also reinstalling the operating system and configuration. We really need someone to step up and take over the role/function/responsibility of Sysadmin, to do this kind of work. --Sysadmin (talk) 07:18, 27 September 2018 (UTC)

I think "a few links/name recognition as a sponsor" is fair enough. I am happy to make a significant donation once the new system is up and running.--Poetlister 19:59, 28 September 2018 (UTC)

As Dysklyver has said already, he is happy to be sysadmin. However, he declines to handle the finance. Really, we need someone in the right country to do that.--Poetlister 16:48, 1 October 2018 (UTC)

Clarification of copyright status and another question

Although I am a Canadian, (from Québec), I know nothing of our Copyrigt laws. I want to upload Lenin, God of the Godless by Ferdinard A. Ossendowski. The book was originally published in 1931 and someone from the USA renewed the Copyright in 1958. Does the book qualify for upload?

Also, I would like to install here my en.Wikisource proofreading tools like Common.css, Common.js and some other .js tools. - I am OK if some won't work, but I don't want to cause a problem.

common.css

common.js

typoscan.js

Thanks in advance for any clarification. Ineuw (talk) 21:45, 22 September 2018 (UTC)

@Ineuw: Copying your preferences is fine. When did the author die? Koavf (talk) 01:00, 23 September 2018 (UTC)

Ferdynand Antoni Ossendowski in 1945. Ineuw (talk) 01:03, 23 September 2018 (UTC)

Do you need the Sysadmin to do anything regarding preferences? I think that this is something each user can do and change as they need individually, let me know if this is otherwise and you need me to do something. --Sysadmin (talk) 05:01, 23 September 2018 (UTC)
Regarding copyright, the standard Canadian law is Life + 50 years, though there are many special exceptions under Canadian law. Generally speaking (if no exception applies) the work you mention is in the Public Domain under Canadian Law. Note that the version you link to is out of an Indian library, where it is life + 60 years. My only question is, why upload the book here if it is already in Archive.org? --Sysadmin (talk) 05:01, 23 September 2018 (UTC)
@Sysadmin: Thanks for asking about my work environment. I have everything necessary. As for uploading Ossendowski's book, is it not the purpose of Biblio.wiki where we can install and proofread books which cannot be uploaded to the Commons/Wikisource because of US Copyright restrictions? Also, the email address selector in Preferences/Notifications leads to a non-existent page. Perhaps my current email address is missing? Ineuw (talk) 03:16, 25 September 2018 (UTC)
@Ineuw: Yes, your purpose to proofread is definitely one we support at Biblio.wiki. For the email functionality, that has not been implemented but definitely needs to be on this Wiki. All the best. --Sysadmin (talk) 04:07, 26 September 2018 (UTC)
Thanks.

It looks like we really need to re-host outside of Canada

  • In that case I vote for Belurussia. I don't see special problems with this country, also I can do all neccessary communication with provider, if needed. They speak Russian and English also will be no problem, I think. — VadimVMog (talk) 05:22, 2 October 2018 (UTC)
  • OK , but could you point out some good provaiders in this country? Electron   10:06, 2 October 2018 (UTC)
  • I'll make a small research, talk with some Belorussians. — VadimVMog (talk) 15:54, 2 October 2018 (UTC) By the way, please remind me what functionality a provider must have? — VadimVMog (talk) 16:18, 2 October 2018 (UTC)
I'll just point out that the country is now usually called Belarus in English. Simon Peter Hughes (talk)
It's because in Soviet times it was Belorussia, now it is Belorus, and I still call it Belorussia out of habit. It's difficult to remember all those renames, please forgive me. I need some information: 1) how much space we need 2) how much traffic our site produces monthly 3) what about wiki-engine? provider must have it? — VadimVMog (talk) 06:01, 3 October 2018 (UTC)
I am not an expert but some information are provaded in the previous topic, especially -> Project:Community_Portal/en#Bibliowiki hosting. What above space: @Sysadmin: says that 80gb will be enough at the moment. Electron   08:57, 3 October 2018 (UTC)
On the second thought, Belarus is not the best choise exactly for me. US can't push Belarus, but Russia can. So it may happen, that Russian content will be the reason for pressure. What other options beside Belarus do we have? Interesting to me look China Cuba, Hong Kong, Indonesia, Iran, Kazakhstan (is more independent than Belarus), Korea, Malaysia(?), Monaco, Mongolia, Philippines, South Africa, Taiwan, Thailand, Vietnam. Don't know much about Egypt. The full list is hereVadimVMog (talk) 05:45, 4 October 2018 (UTC)
indonesia is not an option. In Indonesia, copyright expires 50 years after publication for films and photographs. But for literary works and music, the copyright term is life +70 years. Simon Peter Hughes (talk) 16:13, 5 October 2018 (UTC)
width 400px
For countries which have decent Internet infrastructure and Life+50 there are a few options to consider: Belarus, South Korea, and Hong Kong. While there are many countries in Southeast and East Asia with Life+50, most are not viable because of either a lack of Internet infrastructure/policies and/or local censorship laws and/or practices.
Belarus is the only CIS country in Europe that is Life+50, so they have clearly taken a pro-public domain stance and there are no signs of any changes coming. I agree they are potentially pressurable by Russia, but that hasn't happened yet, including the CIS agreements where everyone else converted to Life+70. The only negatives are that I think VPNs are illegal (which is what I usually use to communicate/manage servers with), and VPS hosting will be more expensive than elsewhere (though still reasonable).
Hong Kong is Life+50 and previously had UK laws, which make them more legally clear, though it is increasingly coming under censorship from China. Personally I think that China is a greater threat to Hong Kong (though they also support Life+50) than Russia is to Belarus.
South Korea is Life+50, and they were never invited to be a part of TPP, though they have many/enough trade agreements with other countries. They are likely the most independent of the three countries listed. In addition, Amazon has AWS Lightsail with a Seoul, South Korea location. This provides an inexpensive VPS hosting option --> 2gb ram/60gb ssd for $10/mo.
Japan is Life+50 but they have a provision for WWII Allies countries where from 8-11 years are tacked on because of the lack of copyright protections during the war. The exact length depends on the country and when they signed onto an agreement, so it is all a bit complex. Treat them as a Life+60 country and that is likely the most accurate assessment (in which case they resemble India in copyright length). Also, Japan was about to cave into Life+70 demands in the TPP agreement (along with MY, NZ, VN) until that provision was pulled from the agreement with the leadership of Canada. --Sysadmin (talk) 05:32, 9 October 2018 (UTC)
Based on research of Sysadmin I would go South Korea. If in a few years the country joins TPP, we can try Belarus. — VadimVMog (talk) 05:27, 12 October 2018 (UTC)
  • Based on the above I have got Jeremy to fire up an instance in Seoul, as November is fast approaching. Currently it is accessible at wikilivres.org. This will allow things to progress since I can the wiki up and people can register however this still leaves the issue of things like the DNS and the issue of actually moving the content over. I can easily move the content with a bot, however this does not retain revision history. xml dumps are better. Dysklyver (talk) 21:01, 22 October 2018 (UTC)
This is great! What do you need me to do? Contact via email: info@biblio.wiki regarding DNS, any admin access to the current system, etc.? Also, if someone needs to deal with the ongoing donations collection and invoice payment. --Sysadmin (talk) 14:18, 25 October 2018 (UTC)

Comment - South Africa would also be a good place to look at (as a fallback, now that we seem to be sorted). pma-50 (unless it's changed?), not as susceptible to us pressure on trade (they're far more involved in african-internal trade), & ther is a significant open-source culture there. the guy who created ubuntu (operating system), mark _? (can't google right now, will fill it in later) is from s.a. Lx 121 (talk) 21:12, 2 November 2018 (UTC)

They do still have a life+50 copyright term, but Amazon does not offer services there yet. Arthur (talk) 18:28, 6 November 2018 (UTC)

Account migration

It looks like my edits, user rights, preferences, etc. have all been reset. Do we anticipate that they will be integrated in December at the .org domain? Koavf (talk) 01:20, 2 November 2018 (UTC)

Ok I should probably explain exactly what is going on;
  1. The wikilivres.org domain is new, however the current plan is to continue using the biblio.wiki domain. Either way both domains will end up aliasing to the same server along with the wikilivres.ca domain. There are no plans as yet to acquire any .kr domains but that is the country tld for South Korea.
  2. The server is not being moved as such, a more accurate description would be that we are building a completely new wiki and then importing the same page content and files. It is not possible to copy server logs, user accounts, passwords, preferences, or watchlists.
  3. This also means anything that was deleted will be effectively hard deleted as part of the move.
  4. I may have to convert a lot of the images to newer file formats, there are some weird files to deal with.
  5. The wiki will have special character support, and potentially support math. If anyone thinks it will be useful I can install Visual Editor. Additionally the wiki will support lua scripts, and several other more modern features that Wikipedia uses, such as per page custom CSS styling and moderation.
  6. The new wiki setup will use Nginx and thus be faster.
  7. There will keep a live backup of the wiki on a seperate server (which hosts other wikis) in case of emergency, the backup will stay up to date by means of a bot. The backup will ensure that the wiki does not get lost.
  8. No bots will be given any deletion powers because this can cause major headaches. You will note that admins cannot delete or protect pages or block people. This is by design.
  9. The wiki will have sendmail and thus support emails.
  10. Editors may potentially be able to have @wikilveres.org or @biblio.wiki email addresses if there is interest in this.
  11. Potentially, anonymous editing could be allowed since unmoderated edits are not seen.
  12. If you haven't already noticed, my email address is dysklyver@linuxmail.org or for (current, not ongoing) hosting queries there is Jeremy at jl22388@ou.ac.uk who will also be hosting the backup going forward. The previous sysadmin will still be involved and is still reachable at info@biblio.wiki
  13. I expect that things will be fully back up and running in about a week, two at the most. By the start of December all technical issues should be resolved.
Arthur (talk) 13:45, 2 November 2018 (UTC)

P.S. I will assign anyone that asks the same user-rights as before and do particularly want to have at least three active crats. Arthur (talk) 13:52, 2 November 2018 (UTC)

Yo, i go away for a while & everything happens? xD glad we are still alive & kicking though! (& did we ever have 3 active 'crats?) Lx 121 (talk) 21:07, 2 November 2018 (UTC)

The list of previous admins and bureaucrat is here: Project:Administrators. If admins cannot delete any pages who will do this? Some texts can be not useful or suitable here (or with bad title) or they even can brake copyrights... And how we will deal with them? Electron (talk) 00:57, 4 November 2018 (UTC)
I will probably have to make a "deleter" user-right to deal with issues that can't be dealt with by blanking or pagemoves. Adminbots are useful but I don't want bots able to delete stuff. Page protection/edit filters/captchas/spamblacklist/blocking are simply not needed in combination with the moderation tool. Sysadmin (talk) 10:16, 4 November 2018 (UTC)

I am happy to become a 'crat. I have plenty of relevant experience.--Poetlister 19:31, 4 November 2018 (UTC)

If you think it would be helpful I can be a bureaucrat as I was in the past. Electron (talk) 12:23, 5 November 2018 (UTC)

  • Ok you are both crats now. For reference the "staff" user-right is functionally similar to a combination of admin/crat and bot. Arthur (talk) 17:32, 6 November 2018 (UTC)

"Awaiting Moderation"

what's with the "awaiting moderation/pending review" thing? o__0 Lx 121 (talk) 21:12, 2 November 2018 (UTC)

It's an anti-spam tool. Anyone can edit but their edits can't been seen until they are approved (via Special:Moderation). Anyone that isn't a vandal can be added to the auto-moderated usergroup and all their edits are automatically approved. It's good because vandalism doesn't need to be reverted, will never affect a page or show up in recentchanges, can be approved hours or even days later, and the system doesn't affect anyone that isn't a vandal. It also means that someone can try and vandalise the wiki it will have no visible effect so they give up pretty fast. Sysadmin (talk) 16:10, 3 November 2018 (UTC)
ok, i get that & i can see its usefulness, but: a) can i be approvced, please? xD [it looks like i have been "approved"? & ty ifso :)] & b) that is going to make more work for the admins (or whoever is going to get the user rights to approve edits). do we really have that much trouble with vandals making edits on existing pages? (as opposed to new page creation?) o__0 & could we exempt talkpages & portal/discussion/whatever -pages from this requirement? because that's going to just clog discussions like crazy, if we ever do get new people coming here... Lx 121 (talk) 02:49, 4 November 2018 (UTC)
I could exempt talk pages from moderation I guess. Moderation does also cover page creation and even stops user account creation showing in recent changes. The issue of course is that if a vandal comes along and over an hour replaces 500 pages with 100k "Lorum ipsum" text (this is a common thing) then no-one logs on for a few days, then hundreds if not thousands of readers will be rather confused. Sysadmin (talk) 10:16, 4 November 2018 (UTC)
There is currently a lot of spammers trying to post ads. I will see if this continues before relaxing the anti-spam filter, in the meantime just give auto-moderated status to anyone that isn't a spambot. It occured to me I could create a draft namespace which isn't moderated and also not indexed like Wikipedia does. Arthur (talk) 17:36, 6 November 2018 (UTC)
  • Moderation is disabled in user and project talk namespaces. New users can't create pages until they are autoconfirmed, after one edit and ten minutes of being registered. Sysadmin (talk) 21:16, 6 November 2018 (UTC)

Update

A brief update on how things are going:

  • There are Echo (notifications), Thanks, and Wikilove extensions installed. These will be familiar to people from Wikimedia sites.
  • The site uses Google analytics and Piwik analytics to monitor the number of users on the site at any one time. This tracks basic details like your country, device type, and whether you are using the mobile or desktop site.
  • There is now a working checkuser extension. Admins can view the log at Special:CheckuserLog however since no-one has the checkuser permission that log will hopefully stay empty. (The checkuser extension is switched off at backend now.)

Selected files will be uploaded from the old server, with the following caveats:

  • If the image can plausibly be hosted on commons, it should be hosted on commons.
  • Icons should not be uploaded here, find a new icon on Wikimedia commons if needed. This is an advantage since the icon will be fetched from Wikimedia's faster CDN and also not cut into our data transfer allowance.
  • Uploading a 50mb Djvu file uses slightly over 500mb of server space. There is around 40gb of server space for files. This is potentially enough for 80 works.
  • The new server will not be able to host .mid or .ogg files. I will host these separately on a CDN and enable an interwiki link to make them downloadable. Any file that is not actively displayed by the mediawiki software, for example an epub or kindle ebook of a fully transcribed file, can be hosted in this way.

I will also make the following point clear: Deleting something does not remove it from the server, when deleting an image that should not be on the server please make sure to tell me about it so I can actually remove it from the server.

Arthur (talk) 17:59, 6 November 2018 (UTC)

Fundraiser

There is no obligation to donate, indeed I don't expect anyone to :D

To hide/not see the fundraiser simply add;

.fundraiser { display:none }

To your personal css in your user preferences. Sysadmin (talk) (Jeremy) 12:14, 8 November 2018 (UTC)

If anyone ever wants to consider affiliate links I made Template:Amazon for the purpose. Linking to Amazon is a kind of an advert and I do not advocate using it without good reason. Just saying. Jl22388 (talk) 16:52, 10 November 2018 (UTC)

Uploads

The server can now support single uploads of 50mb. Remember that a 50mb Djvu file will grow to around 500mb with thumbnails. Sysadmin (talk) (Arthur) 14:42, 9 November 2018 (UTC)

Thanks for revival

Have not visited wl/bibliowiki for a long time, and now am happily seeing the new site. Thanks to all who worked and are working on it. K67y (talk) 01:39, 10 November 2018 (UTC)