Re: Availability of Wikidata JSON dumps after Feb, 2019

--===============7294206084741930293==
Content-Type: multipart/alternative; boundary="00000000000023356705b4ee6b99"

--00000000000023356705b4ee6b99
Content-Type: text/plain; charset="UTF-8"

Hi Daniel,

I am the one managing the archival process and indeed, it was around
end-2018 when the archival process just died (you can see the status here:
https://dumps.wmflabs.org/status.php).

The current status is that the software behind the archival process is
being reworked and will come with features that I will be announcing once
it is ready. The Wikidata JSON dumps will resume archival starting next
week, so unfortunately all information between end-2018 till around October
2020 will be lost (unless someone has a copy somewhere). As for the dumps
in 2017, there were other issues that caused the archival process to stall
as well (you can see the list of available and archived dumps here:
https://dumps.wmflabs.org/wikidata.txt).

I sincerely apologize for the lost information. The new version that I'm
currently working on right now will definitely be much better and more
robust to handle failures.


Warmest regards,
Hydriz


On Wed, 25 Nov 2020 at 20:22, Daniel Garijo <[email protected]> wrote:

> Hello,
>
> I am writing this message because I am analyzing the Wikidata JSON dumps
> available in the Internet Archive and I have found there are no dumps
> available after Feb 8th, 2019 (see
>
> https://archive.org/details/wikimediadownloads?and%5B%5D=%22Wikidata%20entity%20dumps%22).
>
> I know the latest dumps are available at
> https://dumps.wikimedia.org/wikidatawiki/entities/, but unfortunately
> they only cover the last few months.
>
> I also noticed some gaps in the years where there are JSON dumps
> available. For example, there are no JSON dumps available between end of
> Feb, 2017 and Aug 21st, 2017; or between August 21st, 2017 and Nov 16,
> 2017.
>
> Another strange finding is that while there are some entries for the
> dumps in the Internet Archive between March 19th, 2018 and Nov 26th,
> 2018 (e.g., https://archive.org/details/wikibase-wikidatawiki-20181104),
> none of them contain a JSON dump. That's another gap of more than 8 months.
>
> Does anyone on this list know where some of these missing Wikidata dumps
> may be found? If anyone has pointers to a server where they can be
> downloaded, I would highly appreciate it.
>
> Thanks in advance,
> Daniel
>
>
> _______________________________________________
> Xmldatadumps-l mailing list
> [email protected]
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>


-- 
Hydriz Scholz

--00000000000023356705b4ee6b99
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Daniel,<br><br>I am the one managing the archival proce=
ss and indeed, it was around end-2018 when the archival process just died (=
you can see the status here: <a href=3D"https://dumps.wmflabs.org/status.ph=
p">https://dumps.wmflabs.org/status.php</a>).<br><br>The current status is =
that the software behind the archival process is being reworked and will co=
me with features that I will be announcing once it is ready. The Wikidata J=
SON dumps will resume archival starting next week, so unfortunately all inf=
ormation between end-2018 till around October 2020 will be lost (unless som=
eone has a copy somewhere). As for the dumps in 2017, there were other issu=
es that caused the archival process to stall as well (you can see the list =
of available and archived dumps here:=C2=A0<a href=3D"https://dumps.wmflabs=
.org/wikidata.txt">https://dumps.wmflabs.org/wikidata.txt</a>).<br><br>I si=
ncerely apologize for the lost information. The new version that I&#39;m cu=
rrently working on right now will definitely be much better and more robust=
 to handle failures.<br><br><br>Warmest regards,<br>Hydriz</div><br><br><di=
v class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, 25 No=
v 2020 at 20:22, Daniel Garijo &lt;<a href=3D"mailto:[email protected]">dgari=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">Hello,<br>
<br>
I am writing this message because I am analyzing the Wikidata JSON dumps <b=
r>
available in the Internet Archive and I have found there are no dumps <br>
available after Feb 8th, 2019 (see <br>
<a href=3D"https://archive.org/details/wikimediadownloads?and%5B%5D=3D%22Wi=
kidata%20entity%20dumps%22" rel=3D"noreferrer" target=3D"_blank">https://ar=
chive.org/details/wikimediadownloads?and%5B%5D=3D%22Wikidata%20entity%20dum=
ps%22</a>). <br>
I know the latest dumps are available at <br>
<a href=3D"https://dumps.wikimedia.org/wikidatawiki/entities/" rel=3D"noref=
errer" target=3D"_blank">https://dumps.wikimedia.org/wikidatawiki/entities/=
</a>, but unfortunately <br>
they only cover the last few months.<br>
<br>
I also noticed some gaps in the years where there are JSON dumps <br>
available. For example, there are no JSON dumps available between end of <b=
r>
Feb, 2017 and Aug 21st, 2017; or between August 21st, 2017 and Nov 16, 2017=
.<br>
<br>
Another strange finding is that while there are some entries for the <br>
dumps in the Internet Archive between March 19th, 2018 and Nov 26th, <br>
2018 (e.g., <a href=3D"https://archive.org/details/wikibase-wikidatawiki-20=
181104" rel=3D"noreferrer" target=3D"_blank">https://archive.org/details/wi=
kibase-wikidatawiki-20181104</a>), <br>
none of them contain a JSON dump. That&#39;s another gap of more than 8 mon=
ths.<br>
<br>
Does anyone on this list know where some of these missing Wikidata dumps <b=
r>
may be found? If anyone has pointers to a server where they can be <br>
downloaded, I would highly appreciate it.<br>
<br>
Thanks in advance,<br>
Daniel<br>
<br>
<br>
_______________________________________________<br>
Xmldatadumps-l mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Xml=
[email protected]</a><br>
<a href=3D"https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l" rel=
=3D"noreferrer" target=3D"_blank">https://lists.wikimedia.org/mailman/listi=
nfo/xmldatadumps-l</a><br>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
 class=3D"gmail_signature">Hydriz Scholz</div>

--00000000000023356705b4ee6b99--


--===============7294206084741930293==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWG1sZGF0YWR1
bXBzLWwgbWFpbGluZyBsaXN0ClhtbGRhdGFkdW1wcy1sQGxpc3RzLndpa2ltZWRpYS5vcmcKaHR0
cHM6Ly9saXN0cy53aWtpbWVkaWEub3JnL21haWxtYW4vbGlzdGluZm8veG1sZGF0YWR1bXBzLWwK

--===============7294206084741930293==--