Removing MWI -- we were added from herald rules related to JobQueue.

In T54647#10897742, @tstarling wrote:

46% of Googlebot requests use the smartphone User-Agent, and these mostly get a 302 redirect to commons.m.wikimedia.org. But the mobile site gives a link rel=canonical pointing back to commons.wikimedia.org, i.e. the redirect it just crawled.

In T54647#10897816, @tstarling wrote:

In T54647#10891231, @Pigsonthewing wrote:

There is some guff about retail use cases in its introduction, but it is not explicitly required. Indeed it also gives the use case of "a manufacturer site that's not commerce enabled, strictly informational)" where the ''optional'' inclusion of a value for price is to be "assumed that the price attribute represents the manufacturer's suggested retail price. "

46% of Googlebot requests use the smartphone User-Agent, and these mostly get a 302 redirect to commons.m.wikimedia.org. But the mobile site gives a link rel=canonical pointing back to commons.wikimedia.org, i.e. the redirect it just crawled. This is probably the biggest issue in terms of crawl budget. Mobile domain sunsetting (T214998) is already in progress and presumably will fix it; the SEO aspect of that change was discussed on mediawiki.org.

In T396168#10891191, @Pigsonthewing wrote:

they abuse the hProduct microformat

Do they? Or does Google fail to understand it?

Wikipedia is also using microformats incorrectly.

they abuse the hProduct microformat

CommonsMetadata extensions outputs the ImageObject JSON+LD for images, but not the VideoObject for our video pages. That might be an improvement. Additionally, the open graph for all our pages designates type:website, which might reduce the chance that Google is able to recognize this as a video watch page as well I'm guessing.

I documented the history of sitemaps on Wikimedia websites.

I split out the video issue to T396168: Commons videos not indexed by Google

I understand the need to have multi write backends and doing all write operations in both dcs but from that it requires a massive leap to require practically every operation be replicated in both dcs. For example, why doing consistency check in both dcs? let's say something is corrupted in between these two swift instances, the chance of the swift reconciliation script actually finding it or overwriting the secondary dc is much higher than someone accidentally deciding to upload a new version and then getting broken. i.e. I think mediawiki at the moment should be responsible for double uploads (and other write operations) but it shouldn't try to do integrity checks of two swift clusters (doubly so during upload). To me it's like mediawiki checking primary database and a replica for data integrity during page reads, worse than that even. It tries to do that while the replica is thousands of kilometers away. File backend shouldn't do the work of the infrastructure at run time.

In T54647#10887505, @Pigsonthewing wrote:

The class "vevent" is part of the vCalendar microformat [1]. The class is widely used in Wikimedia projects, so please be careful—and check whether it was intended or "blindly copied"—before removing it elsewhere.

Change #1114082 abandoned by Jdlrobson:

[mediawiki/core@master] DataTable: Fit data tables to content to avoid overflowing sidebar

Reason:

Abandoning. Might come back to this in September-time.

https://gerrit.wikimedia.org/r/1114082

@sbassett Could you make this public? Thanks.

In T54647#10374890, @TheDJ wrote:

One other thing I realised today, Google hates duplicate content, and due to how Commons description pages are proxied to the 'local' wiki's, they might be downlinked for being duplicate content ?

In T54647#10887076, @tstarling wrote:

One of the problems with relying on category pages for discovery (assuming that's what's happening) is that the "next page" link on paginated categories is blocked by robots.txt. For example https://commons.wikimedia.org/w/index.php?title=Category:Photographs_taken_on_2017-07-18&from=Z

74 videos indexed. 482K videos not indexed, reason "Video isn't on a watch page".

In T54647#10886855, @LennardHofmann wrote:

Can you try this on https://commons.wikimedia.org/wiki/Category:M%C3%B6nchenholzhausen again? The warnings should have been fixed by this and that edit.

In T54647#10886592, @tstarling wrote:

I have Google Search Console access for Commons. Here is a result for a random URL:

I have Google Search Console access for Commons. Here is a result for a random URL:

Change #1152791 merged by jenkins-bot:

[mediawiki/extensions/TimedMediaHandler@master] Improve HTML escaping in getLongDesc(), getShortDesc() methods

https://gerrit.wikimedia.org/r/1152791

Change #1152790 merged by jenkins-bot:

[mediawiki/core@master] Improve HTML escaping in getLongDesc(), getShortDesc() methods

https://gerrit.wikimedia.org/r/1152790

The idea of preloadFileStat() was to allow concurrent HEAD requests to a list of objects after an relevant locks were acquired. If no locks are acquired, and "latest" is not set, maybe reusing prior loaded state entries is OK. From the perspective of FileBackend, it's mostly thinking that you call doOperations or doQuickOperations, which is supposed to do one preload (within any locking) and is done. The FileBackendMultiWrite class (itself a hack due to not having a proper regional swift cluster and swift-repl only able to do periodic reconciliation) also has to write to the remote backend and has consistency checks turned on...doing a preloads of local and remote backend. It also has to repeat the write operation on the remote backend, requiring another preload to the remote. Since FileBackendMultiWrite does it's own locking, it seems like a lot of these 'stat' entries could be reused instead of reloaded.

Okay, I made deeper investigation. I uploaded a random file on verbose mode and here is the result:

• It made more than 30 HTTP requests to swift: https://logstash.wikimedia.org/goto/8da6c87a8ed733b9a384b238e070e66e out of them, 14 are to the remote datacenter: https://logstash.wikimedia.org/goto/4061ac5482debab31857cf7d1894d04f
• It is clearly making the same requests multiple times back to back. For example, look at 14 calls to remote datacenter. HEAD (which is called from ::getContainerStat() or ::doGetFileStatMulti()) has been called on the same path back to back before the calls to PUT (there are more after it)
  • e.g. https://logstash.wikimedia.org/app/dashboards#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.06.05?id=IWR2PZcBeu0zMOTS9dTJ and https://logstash.wikimedia.org/app/dashboards#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.06.05?id=gtp2PZcBc9IpmxE59lGF are duplicates of each other
  • And https://logstash.wikimedia.org/app/dashboards#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.06.05?id=5WR2PZcBeu0zMOTS9NKi and https://logstash.wikimedia.org/app/dashboards#/doc/logstash-*/logstash-mediawiki-1-7.0.0-1-2025.06.05?id=g9p2PZcBc9IpmxE59lGF also duplicate of each other
  • And this is just the remote dc, the local dc is the same story.
• So there are two major issues:
  • FileBackendMultiWrite::getFileStat() checks for read backend which with the option of readAffinity set in production, it should only read from the local swift. This is clearly not happening.
  • It is making the same expensive call again and again.

In T395834#10882383, @tstarling wrote:

Can you update the doc comments of the File and MediaHandler methods to indicate that it is unsafe HTML and Sanitizer::removeSomeTags() should be called on the result?

Change #1153710 merged by jenkins-bot:

[mediawiki/core@REL1_44] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153710

Change #1153711 merged by jenkins-bot:

[mediawiki/core@REL1_43] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153711

Change #1153712 merged by jenkins-bot:

[mediawiki/core@REL1_42] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153712

Change #1153713 merged by jenkins-bot:

[mediawiki/core@REL1_39] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153713

Change #1153713 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@REL1_39] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153713

Change #1153712 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@REL1_42] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153712

Change #1153711 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@REL1_43] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153711

Change #1153710 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/core@REL1_44] Treat File::getShortDesc() as possibly unsafe HTML

https://gerrit.wikimedia.org/r/1153710