< Architecture meetings

22:00-23:00 UTC on 30 July 2014, at #wikimedia-office connect.

Requests for Comment to review

  1. Requests for comment/CentralNotice Caching Overhaul - Frontend Proxy

Summary and logs


Meeting summary

  • LINK: https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-30 (sumanah, 22:00:19)
  • Future RFC stuff (sumanah, 22:00:38)
    • Please check out https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap - Brad Jorsch is updating it 'as "notify people of upcoming changes", "chance for people to object to upcoming changes", and "todo list" all at once.' (sumanah, 22:00:47)
    • A reminder that I'm going on vacation in mid-August, and will take that opportunity to stop running these meetings and stop coordinating the process overall. I'm working with Rachel Farrand, Quim Gil, and the architecture committee to make sure stuff still happens. (sumanah, 22:01:15)
    • Next week there is no RfC meeting on IRC because of Wikimania. Are there specific topics that you want to discuss in an architecture meeting at Wikimania? (sumanah, 22:01:31)
    • marktraceur: I personally will be running the meeting on Aug 16 and then I will stop forever. I am working with others to ensure that the process keeps going. It may change. (sumanah, 22:02:25)
  • CentralNotice Caching Overhaul - Frontend Proxy RfC (sumanah, 22:04:37)
    • Today we're talking about Matt Walker's proposal to "have a static javascript blob, injected via skin hook, in every content namespace page that will call up to a special CentralNotice banner server". This has frontend caching implications. (sumanah, 22:04:43)
    • LINK: https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy (sumanah, 22:05:04)
    • The RFC has evolved in the last few weeks. As Matt explained just before he updated it: "I'm going to unify the URL to bits.wikimedia.org; varnish will initially serve the request before forwarding that to a backend; I'll add a bit to talk about CSP and how I dont currently care about it w/ justification that the first load is REALLY important; I'm going to add a note on where the 200GB comes from" (sumanah, 22:05:31)
    • this is a followup to the meeting in October 2013 https://www.mediawiki.org/wiki/Talk:Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy#IRC_meeting_2013-10-02_34156 (sumanah, 22:05:58)
    • Matt is about to leave the Foundation so in today's meeting we want to clear up any Ops concerns so that Anne Gomez (Fundraising tech product manager atgo ), ejegg, Jeff_Green, AndyRussG, and others can carry on implementing it after Matt's gone (sumanah, 22:06:20)
    • discussion of ESI solution (sumanah, 22:29:06)
    • ESI is very broken with our currently deployed varnish; expected to be working for 4 upgrade but thta’s a ways off (brion, 22:30:49)
    • rfc proposal doesn’t fix “page bump” when banners are loaded, but reduces its delay by 1 RTT between client and server (brion, 22:32:27)
    • using JS blob now means a cache clear required for ESI transition later, but this bblack thinks it may be managable (especially if can be done gradually) (brion, 22:38:11)
    • alt proposal to put the banner code (node-backed?) directly on the main domain, bypassing the cookie bootstrapper js blob. use <script src async> (brion, 22:43:16)
    • we think <script src async> in <head> is ok. we think. (brion, 22:45:52)
    • per mwalker we can do the load below RL for added safety to avoid blocking (brion, 22:46:58)
    • AGREED: * hosting the banner generator on the main wiki domains (via a varnish proxy config?) (brion, 22:50:12)
    • AGREED: * using a consistent URL for those in <script src=“” async> (brion, 22:50:20)
    • AGREED: * possibly moving that script around to minimize issues (brion, 22:50:28)
    • AGREED: * with a general thought that ESI will be better someday, and we can then migrate towards using the same generator via ESI instead of JSONP (brion, 22:50:40)
    • ACTION: mwalker to revise RfC in agreed-upon directions (brion, 22:50:51)
    • need to know who’s point person once mwalker is gone (brion, 22:51:27)
    • LINK: https://www.mediawiki.org/wiki/Performance_guidelines#Caching_layers (AndyRussG, 22:52:27)
    • ACTION: post-mwalker point person yet to be determined (brion, 22:52:47)
    • awight & AndyRussG assigned for stuff for the moment, but need folks to check in with on details :D (brion, 22:55:18)
    • and Roan can help with frontend caching & JS interaction (brion, 22:55:32)


Meeting ended at 22:55:49 UTC.

Action items

  • mwalker to revise RfC in agreed-upon directions
  • post-mwalker point person yet to be determined


Action items, by person

  • mwalker
    • mwalker to revise RfC in agreed-upon directions
    • post-mwalker point person yet to be determined

Full log

Meeting logs
22:00:06 <sumanah> #startmeeting CentralNotice Caching Overhaul - Frontend Proxy | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours
22:00:06 <wm-labs-meetbot`> Meeting started Wed Jul 30 22:00:06 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:06 <wm-labs-meetbot`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:06 <wm-labs-meetbot`> The meeting name has been set to 'centralnotice_caching_overhaul___frontend_proxy___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours'
22:00:12 <sumanah> #chair sumanah brion mark
22:00:12 <wm-labs-meetbot`> Current chairs: brion mark sumanah
22:00:19 <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-30
22:00:21 <brion> hey all
22:00:24 <ori> hello
22:00:27 <sumanah> Hi!
22:00:34 <sumanah> First off - about next week & next month
22:00:38 <sumanah> #topic Future RFC stuff
22:00:47 <sumanah> #info Please check out https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap - Brad Jorsch is updating it 'as "notify people of upcoming changes", "chance for people to object to upcoming changes", and "todo list" all at once.'
22:00:57 <sumanah> (Brad isn't here right now)
22:01:11 <marktraceur> Aw, anomie...
22:01:12 <sumanah> (API changes! ever looming!)
22:01:15 <sumanah> #info A reminder that I'm going on vacation in mid-August, and will take that opportunity to stop running these meetings and stop coordinating the process overall. I'm working with Rachel Farrand, Quim Gil, and the architecture committee to make sure stuff still happens.
22:01:31 <sumanah> #info Next week there is no RfC meeting on IRC because of Wikimania. Are there specific topics that you want to discuss in an architecture meeting at Wikimania?
22:01:48 <marktraceur> sumanah: "stop running these meetings" forever? Or just for a while?
22:02:25 <sumanah> #info marktraceur: I personally will be running the meeting on Aug 16 and then I will stop forever. I am working with others to ensure that the process keeps going. It may change.
22:02:32 <marktraceur> KK
22:03:01 <sumanah> Re Wikimania: who here is going? do you want to talk about any specific architecture/RfC stuff there? I know Daniel Kinzler suggested talking about "Using factory functions for component instantiation" there
22:03:21 <matanya> o/
22:03:59 <sumanah> well, we don't have to discuss it here - "[Wikitech-l] Anyone going to WikiMania?" will do
22:04:12 <sumanah> hey atgo :)
22:04:23 <atgo> hey sumanah !
22:04:27 * awjr waves
22:04:36 <sumanah> atgo: you are just in time! because now that those announcements are over:
22:04:37 <sumanah> #topic CentralNotice Caching Overhaul - Frontend Proxy RfC
22:04:43 <sumanah> #info Today we're talking about Matt Walker's proposal to "have a static javascript blob, injected via skin hook, in every content namespace page that will call up to a special CentralNotice banner server". This has frontend caching implications.
22:05:04 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy
22:05:26 <sumanah> I'm now gonna summarize for mwalker - say if I'm wrong or omitting something important here Matt :)
22:05:31 <sumanah> #info The RFC has evolved in the last few weeks. As Matt explained just before he updated it: "I'm going to unify the URL to bits.wikimedia.org; varnish will initially serve the request before forwarding that to a backend; I'll add a bit to talk about CSP and how I dont currently care about it w/ justification that the first load is REALLY important; I'm going to add a note on where the 200GB comes from"
22:05:58 <sumanah> #info this is a followup to the meeting in October 2013 https://www.mediawiki.org/wiki/Talk:Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy#IRC_meeting_2013-10-02_34156
22:06:20 <sumanah> #info Matt is about to leave the Foundation so in today's meeting we want to clear up any Ops concerns so that Anne Gomez (Fundraising tech product manager atgo ), ejegg, Jeff_Green, AndyRussG, and others can carry on implementing it after Matt's gone
22:06:33 <mwalker> one implementation point up front -- right now the JS is injected via skin hook because we're proposing to host this on bits. if instead we agree to host it on some wiki url, e.g. en.wm.o/banners than it can be a simple <script src=""> injection rather than a blob. Reason being is because I need cookies that are stored on the wiki domain
22:06:57 <mwalker> *or rather, on all wiki urls so that it's "local" to the wiki
22:07:19 <sumanah> bblack: chasemp mark: Jeff_Green ori - this is definitely a good time for us to talk about Ops and performance concerns
22:07:47 <ori> bblack, jgage: what you missed: https://dpaste.de/COB0/raw
22:07:57 <brion> i tend not to like <script src=> because that blocks rendering on a second request; is that expected to be worked around by pipelining or somethign?
22:08:17 <sumanah> atgo: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140730.txt has the announcements in case you want to catch up on those :)
22:08:53 <jgage> thanks ori
22:09:02 <mwalker> brion, I don't currently have pipeline plans beyond what's already in the rfc -- so on page rendering would likely be blocked -- however, my understanding is that if it's in the head then page flow operations can continue?
22:09:30 <atgo> thanks sumanah
22:09:30 <brion> mwalker: alas no, since it could contain a document.write() “anything goes” — it has to literally stop parsing
22:09:42 <brion> unless you async it maybe
22:09:55 <ori> mwalker: the bit about "the first [request] happens in the head and is a call to get the GeoIP country lookup" is no longer accurate, since it's now done by cooking the initial request. does that change things?
22:10:09 <ori> cookie-ing
22:10:21 <RoanKattouw> mwalker: You need to set or read cookies?
22:10:25 <mwalker> brion, I'm not sure how I would async it...
22:10:29 <mwalker> RoanKattouw, I need to do both actually
22:10:37 <mwalker> but on initial request, I only need to read
22:10:42 <RoanKattouw> Surely you can access document.cookie even from foreign origin scripts?
22:10:59 <brion> mwalker: ‘async’ attribute can be used on the <script src=“…” async> i think, though i’m not sure which browsers respect it offhand
22:11:00 <RoanKattouw> Or are you doing server-side cookie reading/writing?
22:11:12 <bblack> I don't have many sophisticated things to say about CentralNotice, but I'd like to point out that BannerRandom requests are at least sometimes very high on our most-hit URLs when looking at Varnish stats, which seems a little troubling (that we spend more hits on that than popular content)
22:11:18 <mwalker> ori, it's inaccurate in so far as that it wouldn't need to do it for every request, but it would still need to do it for ipv6 requests that dont resolve to anything
22:11:37 <mwalker> RoanKattouw, the script on the client can, but the first request up to the server needs to have all the data so that a banner can be selected
22:11:45 <RoanKattouw> Right
22:12:05 <mwalker> if I am just doing a plain <script src=> then that first request wouldn't have the cookies if it was going to a non local domain
22:12:14 <RoanKattouw> Yes
22:12:18 <bblack> A much more egregious non-content URL that tends to sit on top of the stats, but probably isn't in scope here, is /wiki/Special:RecordImpression
22:12:23 <RoanKattouw> So ideally it'd be done locally
22:12:26 <mwalker> if I have the JS blob as in the RfC than the request can go anywhere at the expense of havin gthe JS blob
22:12:39 <brion> mwalker: that sounds wrong unless there’s some CSP stuff i don’t know about going on
22:12:49 <brion> all our JS is served from bits and it has access to document.cookie right?
22:12:52 <RoanKattouw> brion: He needs the /server/ to have the cookies
22:12:54 <mwalker> bblack, that makes sense because bannerrandom gets hit from every page load
22:13:03 <RoanKattouw> I was confused for the same reason just a minute ago
22:13:20 <bblack> mwalker: yeah I just wish there was a way we could not do that, and I'm hoping that's part of this discussion
22:13:21 <brion> but the JS stub is the bit that fetches the cookie from the document.cookie and sends it as a parameter to the server, right?
22:13:24 <ori> mwalker: but it's superfluous to the request log
22:13:43 <brion> so the server doesn’t need the cookie as a cookie, it takes it as a URL query param
22:13:48 <RoanKattouw> brion: Yes, and if we did same-domain hosting then that could be eliminated
22:14:08 <brion> ahhhh i see
22:14:26 <RoanKattouw> mwalker: Ooooh I like your proposed use of frontend Varnishes
22:14:30 <RoanKattouw> (Still reading the RFC)
22:16:20 <mwalker> brion, the real reason I brought up same domain hosting is because the JS blob on page approach prevents us from doing CSP
22:16:30 <brion> *nod*
22:16:39 <ori> is ESI out of the question?
22:16:40 <RoanKattouw> What do you mean by "doing CSP"?
22:16:45 <RoanKattouw> (CSP = content security policy, right?)
22:16:50 <mwalker> RoanKattouw, yes
22:16:59 <bblack> I get that the banner is fairly dynamic based on a lot of things, but could we at least cache it a while (<1hr?) for the user? they don't need new banners on every page load right? they're not that time-critical?
22:17:06 <brion> mwalker: in that case, same-domain hosting with <script src=“…” async> should give about the same behavior as the “JS blob <script>…</script> fetching JSONP” and would indeed work around CSP issues
22:17:24 <mwalker> ori, ESI can work, but I haven't spent a huge amount of time figuring out the best way of doing it
22:17:30 <brion> ESI just scares me :D
22:17:34 <ori> brion: but if it's async would you not still get the page bump?
22:17:48 <RoanKattouw> ori: ESI has only recently been possible now that text is on Varnish, and nothing and no one uses it yet
22:17:50 <bblack> ESI is still kinda borked with our current varnish deployment anyways
22:17:53 <RoanKattouw> So it's a bit scary for those reasons
22:17:56 <RoanKattouw> Oh well and there you go
22:18:00 <brion> ori: yes, but the jsonp fetch is already async so that’s what you’re already getting with this proposal afaict
22:18:06 <bblack> otherwise we'd be using it for the Zero banners
22:18:32 <ori> bblack: how/why is it it borked? (or would answering that carry us too far off-course?)
22:18:42 <brion> if you want to make the jsonp fetch sync, you need to use document.write(‘<script src=“…’)
22:18:43 <RoanKattouw> mwalker: Clarification: is the JSONP request in the head meant to be blocking or async?
22:18:52 <bblack> tl;dr version is it's just borked, upstream, in a way we can't easily fix right now
22:18:53 <RoanKattouw> Because ... yes what brion said
22:19:00 <mwalker> RoanKattouw, I intended it to be async
22:19:08 <csteipp> The "CSP issue" (iirc) is that I'd like to enable CSP that forbids inline scripts.. we're a long way from doing that. We can still do CSP with inline scripts enabled.
22:19:11 <brion> ok
22:19:44 <mwalker> csteipp, that's my understanding as well -- but right now the only inline script I think is ResourceLoader...
22:21:18 <RoanKattouw> RL has some inline scripts yes
22:21:26 <RoanKattouw> It needs to inline certain things for security reasons (irony!)
22:21:34 <RoanKattouw> Well, privacy reasons really
22:22:16 <ori> IMO this needs to be done in Varnish. I realize that ESI is borked, but what mwalker is proposing is not a quick workaround -- it'd require substantial effort to implement, especially with him as the primary architect of this solution not being around
22:22:40 <ori> and Varnish is implicated in this setup by dint of already having to do work on each request to check the GeoIP cookie and perform geolocation as necessary
22:22:48 <ori> and we don't have a suitable alternative for that
22:23:02 <mwalker> NodeJS can use the same GeoIP libraries we already use on varnish
22:23:05 <sumanah> atgo: who is going to be the person working on this in the future, if you know? ejegg? Jeff_Green? AndyRussG? someone else?
22:23:36 <atgo> hey sumanah . should be AndyRussG and awight (he's OOO today)
22:23:43 <ori> and the solution mwalker proposes doesn't (as far as I can tell) solve the page-bump problem
22:24:00 <ori> which the RFC puts forward as one of the very reason this area needs attention
22:24:08 <atgo> but i'm sure awight will catch up on this when he's back
22:24:10 <ori> *reasons
22:24:12 <mwalker> that's correct; we will still have a page bump, but hopefully significantly less of one
22:24:22 <mwalker> *given that we are removing on RTT
22:24:22 <RoanKattouw> mwalker: Wouldn't there be significant benefits in doing same-domain stuff?
22:24:28 <sumanah> atgo: is Adam back by the time mwalker leaves? :)
22:24:29 <RoanKattouw> We'd be able to get rid of the inline script, presumably
22:24:42 <AndyRussG> I'm lurking and learning here BTW
22:24:43 <RoanKattouw> We could potentially even be able to get rid of the page bump
22:24:57 <mwalker> RoanKattouw, that is the one benefit of having same-domain stuff; the one downside is that then we "clutter" the domain root with stuff
22:25:14 <AndyRussG> (Hi all :) )
22:25:18 <mwalker> *actually, another benefit is that on the same domain you dont have to do another DNS lookup
22:25:27 <atgo> sumanah yep! he's not in every day, and today is just one of those days. he'll be back in office friday and checking email between now and then
22:25:45 <sumanah> ok
22:25:50 <atgo> i'll make sure he catches up on this :)
22:25:51 <sumanah> cool, that's good to know :)
22:26:17 <RoanKattouw> If we did something like <script src="/w/banner.php"></script> , and that returned some JS that set some variables and also wrote a CSS rule into the head that does .bannerContainer { height: 247px; } ....
22:26:28 <RoanKattouw> or something like that
22:26:56 <mwalker> ori, if we host this on the same domain and use ESI, we'd still have to have a backend server to create the banner -- not a problem it just means the backend has to do slightly more work -- it will delay the entire page request though
22:27:12 <brion> that fixes the page bump but introduces a delay yeah
22:27:18 <mwalker> RoanKattouw, ah; but I dont want the script to request another something... the downloaded script must have the entire banner to be injected
22:27:27 <mwalker> and we dont know the hight of the banner because that's a browser dependent flow thing
22:27:34 <mwalker> and not all banners are the same height
22:27:45 <brion> do we know what exactly is borked with the ESI solution?
22:27:53 <mwalker> bblack, ^
22:28:17 <RoanKattouw> mwalker: Right, so banners are neither pure images nor pure HTML but a mix of those two?
22:28:52 <sumanah> brion: I need to head off in a few minutes - you'll be finishing up the meeting?
22:28:54 <bblack> it's been a while since we last looked, I couldn't recall details right now. I'm pretty sure most of the problem has to do with us running the upstream-unsupported -plus variant of varnish3 for streaming, but I'm not 100% sure that ESI works correctly even for mainline varnish3
22:28:58 <mwalker> RoanKattouw, *nods*
22:29:02 <ejegg> some even have their own js
22:29:04 <RoanKattouw> Right
22:29:06 <sumanah> #info discussion of ESI solution
22:29:09 <brion> sumanah: confirmed
22:29:18 <RoanKattouw> Yeah then I'm not sure that the bump can be fixed without using ESI
22:29:45 <brion> i tend to agree with RoanKattouw, fixing the bump *and* not introducing a round-trip between client and server requires ESI or something very like it
22:29:49 <brion> bblack: :(
22:29:52 <sumanah> thanks
22:29:54 <AndyRussG> Hmmm banners with js and js with banners
22:29:54 <bblack> but the bottom line is ESI is deeply broken in our currently-deployed varnish, and we've had impetus to fix it already from Zero and not found any easy way to do so, yet. Looking at it was postponed until the varnish v4 upgrade a while back
22:30:09 <RoanKattouw> Unless FR practices significant self-restraint in terms of standardizing banner heights or only using existing blank space or something like that
22:30:18 <mwalker> RoanKattouw, hah!
22:30:32 <mwalker> you can look at this RFC as a stepping stone to ESI... the backend nodejs server would only need small modifications to support it once it works
22:30:37 <RoanKattouw> bblack: Yeah I made that statement not as a "this is why we need ESI now" but more as a "this is why this isn't going to happen soon"
22:30:48 <mwalker> in the meantime, this would reduce page bump by the time of 1 RTT
22:30:49 <brion> #info ESI is very broken with our currently deployed varnish; expected to be working for 4 upgrade but thta’s a ways off
22:30:57 * ori nods
22:31:20 <sumanah> Thanks all, catch you later.
22:31:31 <brion> yeah i don’t hate this solution, but it depends on how much work people think it will be and how much of that is going to be reusable for the ESI better solution later
22:31:43 <brion> i have the feeling that it would be easy to reuse, but i could be wrong
22:32:25 <RoanKattouw> mwalker: So if I understand this all correctly, the JSONP request initiated from the head will return a JSON blob that contains the banner HTML and all the needed data?
22:32:27 <brion> #info rfc proposal doesn’t fix “page bump” when banners are loaded, but reduces its delay by 1 RTT between client and server
22:32:33 <mwalker> RoanKattouw, that's correct
22:32:43 <RoanKattouw> Like, it won't just give us a banner name that we then have to go fetch separately
22:32:45 <RoanKattouw> OK awesome
22:32:46 <RoanKattouw> That's good
22:32:48 <mwalker> and then it'll wait for the RL downloaded centralnotice controller to inject itself
22:32:52 <RoanKattouw> Yeah
22:33:51 <brion> ori: what do you think? would the backend for the jsonp blob be adaptable to a source for ESI banners once ESI is available for doing it “right"?
22:33:57 <RoanKattouw> mwalker: Re CSP if you really want to you can make the head JS an RL top queue module and I think that'll still work. It'll just run a bit later I think, because RL needs to init first
22:34:13 <brion> or would we need to do things more differently twice
22:34:30 <RoanKattouw> Actually I think I would recommend doing that, because what you're currently talking about makes it very difficult to ever change the head JS code
22:34:36 <ori> brion: it sounds like it could be
22:35:37 <ori> bblack: what do you say --- if we went with this approach, is there anything that could be done to make it easier to transition to ESI in the future?
22:35:46 <brion> RoanKattouw: that’s a good point, would hate to have to clear all the caches
22:35:54 <mwalker> RoanKattouw, that's basically what we do right now... and it's slow
22:36:03 <brion> hmmm
22:36:21 <mwalker> *slow because it serializes requests that *could* happen in parallel
22:37:03 <bblack> clearing varnish caches once in a blue moon on a major transition (e.g. to ESI here) isn't *that* huge of a deal, so long as we're ok with the cache clear happening slowly
22:37:19 <bblack> it's different if you need the cache cleared all at once or things are horribly broken for the user
22:37:29 <bblack> (where slowly is like, over a period of a day or three)
22:37:38 <RoanKattouw> mwalker: What does it serialize exactly? Your code vs the other top modules you mean?
22:37:38 <mwalker> hosting it on the same domain solves the *cant change the head js* problem -- you just then cannot change the URL location
22:37:51 <RoanKattouw> Yes that would also wokr
22:38:11 <brion> #info using JS blob now means a cache clear required for ESI transition later, but this bblack thinks it may be managable (especially if can be done gradually)
22:38:14 <mwalker> RoanKattouw, right now it's RL->CentralNotice->BannerRequest->Injection; where it could be RL->Join(CentralNotice, BannerRequest)->Injection
22:38:43 <mwalker> what this RfC proposes is actually Join(RL->CentralNotice, BannerRequest)->Injectiong
22:38:45 <RoanKattouw> (And note that that serializes things just as much, just in the reverse order so the impact is less: RL init waits for CN rather than the other way around)
22:39:21 <RoanKattouw> No, I don't think that's accurate
22:39:33 <mwalker> with that last point, no, if I assume that browsers will execute requests in parallel as they see them
22:39:54 <RoanKattouw> For <script> tags in the <head> that is not true
22:40:00 <RoanKattouw> Oh wait but the code is inlined so there's no request
22:40:02 <RoanKattouw> In any case
22:40:05 <brion> right
22:40:23 <RoanKattouw> What you're proposing is inlining the bootstrapper, then doing Join(RL->CentralNotice, BannerRequest)->inject
22:40:30 <mwalker> RoanKattouw, exactly
22:40:35 <brion> nice
22:40:41 <RoanKattouw> I was proposing something like RL->bootstrapper->Join(CN, BR)->inject
22:41:24 <RoanKattouw> If we go with your other suggestion of hardcoding the URL to the bootstrapper code rather than the bootstrapper code itself, we get bootstrapper->RL->Join(CN, BR)->inject
22:41:24 <brion> that does increase the “time to bump” though doesn’t it?
22:41:46 <RoanKattouw> Well both of those two do
22:41:51 <mwalker> if hosted locally, the bootstrapper code wouldn't exist
22:41:59 <RoanKattouw> Oooh
22:42:01 <mwalker> because NodeJS would have access to the cookies
22:42:08 <mwalker> and so it could just dump the banner data
22:42:19 <RoanKattouw> So you'd do like a <script async> in the head making a request to NodeJS
22:42:23 <mwalker> *nods*
22:42:25 <brion> now we’re cooking with gas
22:42:28 <RoanKattouw> That would be nice actually
22:42:36 <RoanKattouw> Because it's reasonable to ask that URL to stay constant
22:42:46 <RoanKattouw> If we need to change things, we can get NodeJS to output different things
22:43:10 <mwalker> but... I need a local URL on every wiki -- is that a problem?
22:43:16 <brion> #info alt proposal to put the banner code (node-backed?) directly on the main domain, bypassing the cookie bootstrapper js blob. use <script src async>
22:43:27 <RoanKattouw> As far as I'm concerned it's not
22:43:29 <brion> mwalker: that sounds like varnish config should make it easy to do
22:43:45 <RoanKattouw> I don't know whether browsers respect async in the head, I hope they do
22:43:52 <mwalker> its something easy to do; I just didn't know if we wanted to do that; since the only domain with virtual roots right now is bits
22:43:52 <brion> lemme search
22:43:57 <RoanKattouw> But browser debugger tools will let you find out easily
22:44:55 <RoanKattouw> Basically what I'm afraid of is if your <script async> blocks because of browser stupidity, then none of the RL init code will even be *downloaded* until the NodeJS code is done
22:44:56 <brion> RoanKattouw: i think it should be ok, just doesn’t work on inline source scripts
22:45:01 <brion> remote source should be fine i think
22:45:13 <RoanKattouw> Yeah I think it probably will be
22:45:23 <RoanKattouw> But my trust in browsers has eroded :)
22:45:52 <brion> #info we think <script src async> in <head> is ok. we think.
22:46:18 <mwalker> RoanKattouw, we can probably do some fudging to make sure that this request is below RL's blob -- and I think that'll ensure RL is executed first
22:46:58 <brion> #info per mwalker we can do the load below RL for added safety to avoid blocking
22:47:55 <brion> so we’re currently looking at:
22:48:12 <RoanKattouw> mwalker: Actually if async can be shown to work, we should really be doing the reverse
22:48:21 <brion> * hosting the banner generator on the main wiki domains (via a varnish proxy config?)
22:48:34 <brion> * using a consistent URL for those in <script src=“” async>
22:48:38 <RoanKattouw> In any case, test in many browsers and pay close attention to the waterfall graph
22:48:44 <brion> * possibly moving that script around to minimize issues
22:49:09 <brion> * with a general thought that ESI will be better someday, and we can then migrate towards using the same generator via ESI instead of JSONP
22:49:16 <brion> sound about right?
22:49:23 <mwalker> *nods*
22:49:27 <brion> awesome
22:49:39 <brion> any objections to revising the draft in that way?
22:49:48 <RoanKattouw> Sounds good to me
22:50:00 <mwalker> actually; I just remembered why ESI wont work -- ESI will be injecting a JS blob which CSP will then prevent :p
22:50:01 <bblack> looks reasonable to me
22:50:08 <mwalker> but yes; I can revise the RfC for brions list
22:50:12 <brion> #agreed * hosting the banner generator on the main wiki domains (via a varnish proxy config?)
22:50:20 <brion> #agreed * using a consistent URL for those in <script src=“” async>
22:50:28 <brion> #agreed * possibly moving that script around to minimize issues
22:50:40 <brion> #agreed * with a general thought that ESI will be better someday, and we can then migrate towards using the same generator via ESI instead of JSONP
22:50:41 <brion> woot
22:50:50 <mwalker> 10 minutes to spare
22:50:51 <brion> #action mwalker to revise RfC in agreed-upon directions
22:50:55 <brion> awesome :D
22:51:01 <brion> anything else to add or are we about done?
22:51:11 <AndyRussG> whom should I bug for guidance on this once mwalker is gone?
22:51:19 <brion> ah good question
22:51:27 <brion> #info need to know who’s point person once mwalker is gone
22:51:57 <AndyRussG> and do you have any URLs to suggest for reading up on all this infrastructure (beyond anything obvious, anything in the RFC)?
22:52:09 <AndyRussG> sumana sent me one earlier...
22:52:27 <AndyRussG> https://www.mediawiki.org/wiki/Performance_guidelines#Caching_layers
22:52:47 <brion> #action post-mwalker point person yet to be determined
22:52:57 <AndyRussG> *sumanah I meant
22:52:57 <brion> AndyRussG: that’s the main scary stuff i think yeah :D
22:53:28 <brion> ok unless we have an immediate volunteer to take over the project after mwalker i think we’re ready to close
22:53:33 <AndyRussG> I need to learn more about caching for other stuff too :)
22:53:50 <AndyRussG> I mean I think me and awight are more or less assigned to it
22:54:13 <mwalker> that's sort of up to atgo to decide who gets the short stick :)
22:54:29 <brion> spiffy
22:54:30 <AndyRussG> I meant to ask whom to bug with questions when I run into walls
22:54:37 <atgo> yeah that's an awight and AndyRussG thing for the moment
22:54:55 <AndyRussG> I can just ping wildly at anyone who's had the temerity to participate here ^
22:54:56 <atgo> it needs a home, and we're it for the time being :)
22:55:06 <AndyRussG> >_>
22:55:18 <brion> #info awight & AndyRussG assigned for stuff for the moment, but need folks to check in with on details :D
22:55:19 <RoanKattouw> AndyRussG: For frontend caching and interaction with JS and stuff, I can help
22:55:32 <RoanKattouw> Basically for the sort of stuff I talked about in this meeting :)
22:55:32 <brion> #info and Roan can help with frontend caching & JS interaction
22:55:46 <brion> awesome
22:55:47 <AndyRussG> RoanKattouw: thank you!! :)
22:55:47 <brion> ok let’s close this out
22:55:49 <brion> #endmeeting
[22:56:06] <RoanKattouw> (Also I wrote the backend half of ResourceLoader, including caching logic, so I can help with understanding that and with ResourceLoader stuff in general)
[22:56:19] <brion> thanks everybody — this was an enlightening session :D
[22:56:20] <mwalker> AndyRussG: Gwicke or Cscott are probably your bests points of contact for NodeJS,;roan and timo for RL; mark, ori, and bblack for varnish / LVS
[22:56:49] <AndyRussG> cool! thanks
[22:57:15] <mwalker> brion, thanks for moderating the last half
This article is issued from Mediawiki. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.