Meeting 2014 07 14

Agenda

• state of Android
• new contractor (jack)
• Where should the logical vs. physical coordinate boundary be? (SimonSapin)
• XHR progress and next steps (manish) (Unable to dial in)
• 32 build status
• large stacks
• martin's graphics work
• html5 parser
• CI status

Android

• laleh: Currently layout is running, but no text is painted on screen.
• mbrubeck: Same for me. Last time this happened it was because we were passing a bad default scale factor (#2377). Not sure what's causing it this time.
• laleh: We can still do power measurements on the layout code.
• mbrubeck: Script is probably not working, but that may not be as interesting anyway; should be the same as Spidermonkey in Firefox.
• jack: Spidermonkey is expected to be broken because Android is trying to use 64-bit representations on 32-bit hardware. I'm surprised it doesn't crash on startup.
• laleh: Is there 64-bit Android hardware?
• jack: The only shipping 64-bit ARM hardware is Apple's.

contractor

• jack: ms2ger will be working on ServiceWorkers in Servo as a contractor starting this week. They will start with doing a worker design in Servo to do a single WebIDL interface, probably XMLHttpRequest. You will be able to talk to the workers from the main thread and send stuff back. This will require our first cross-origin wrappers. When JS from one page needs to call JS in a different origin, it has to go through a proxy object that makes sure you don't do anything "bad", which should be a fun new piece of code. The ServiceWorker stuff is what most of the debugging and caching stuff is designed on these days. You can hook into all areas of the browser and replace how it downloads resources, etc. It's like round 2 of the caching infrastructure for web apps (since the first round failed miserably). Hopefully, he'll be starting tomorrow officially.

Logical vs. physical coordinate boundary

• SimonSapin: The link is: https://pastebin.mozilla.org/5554709 It's a grep output of all the things in layout code that are logical rect sizes or points in the PR made last week. This builds and runs without regressing horizontal text. But, as soon as we try to use writing modes, it fails at runtime with assertions that we don't mix writing modes, which fails. It appears some of the conversions I made should stay in physical space, but I don't know enough about this code to decide which is which.
• jack: All of this is in layout? I believe that all of layout should be logical. I assumed that layout would be logical, but as soon as we generate display lists for painting, it'd be physical.
• SimonSapin: Yes, but generating them is in the layout tree. Some special cases, like fixed positioning and absolute positioning maybe should be in physical space, but I'm not sure...
• jack: So if you're in a different writing mode, do absolute & fixed positioning work normally, or are they transformed? Because usually you set top, left, etc....
• SimonSapin: Yes, those are physical directions, so they do not change. We may want to map them to logical directions, though, to make the layout code cleaner. We have code for the constraints that says that if one thing is set, we may treat the top property differently. Also, width and height are treated differently. At the end, though, when we store the offset in the page to the element, maybe we want to store that as physical.
• jack: Where are those stored? Probably a 'pub struct...'
• SimonSapin: AbsolutePositionStruct, maybe?
• jack: I feel like I don't have enough info to give concrete advice here. Do you know where the bug is?
• SimonSapin: Maybe I should chat with pcwalton afterwards.
• jack: Also, if you find out where the conversion is missing, it might be easier to talk about how far up the stack that should happen and in what other similar places that should happen, relative to where it does now. It seems like if we only deal with physical coordinates in build_display_list, that would be nice.
• SimonSapin: That's the idea.
• bjz: How will the windowing stuff interact? Like if we're getting cursor position of the mouse, etc.
• mbrubeck: For hit-testing of the cursor?
• SimonSapin: The display items will be in physical space - everything in gfx/. I believe hit testing is on display items.
• jack: It might be now, but that's not how patrick wants it to be in the future? I think in gecko, you take a mouse position and find the DOM node by regenerating the portion of the display list... the reason is you don't want a link from the DOM to the display list. But, that DOM node may be gone because there's a bunch of memory safety stuff that has to be in place. Because when you generate the DOM node, you create stuff in layout, and then by the time the event comes in, you may have had script delete the DOM node.
• zwarich: Another potential problem is in invalidation. If we want to invalidate a region, we'd like to do it without a display list. pcwalton wants to always generate them and diff them, which might solve it, but if you do what blink/webkit do, then we'd have a problem.
• jack: Have you thought about that much? I thought the plan of record was to do display list base invalidation because it works in gecko.
• zwarich: I'm skeptical that generating a display list and diffing it to determine dirty regions is efficient. Today, we just regenerate the whole thing every time, which is fine.
• kmc: We would have incremental display list building, too, right? I think in DLBI, you have to keep nodes around from the old one in order to do it.
• zwarich: Have to keep the old display list around in a form that's amenable to parallelization.
• jack: As you and martin are working on the graphics stack, this is probably a question we'll look to both of your guidance on. Obviously, pcwalton has some opinions, too, but by the time you get through the rewrite, you should have more ideas.

32-bit build & Rust issue tracking

• jack: Has anybody heard something about this fix? Right now, we were running into rust compiler bugs that luqman fixed. I got all that working and got a rust upgrade, and building with optimizations, it works great. But if you build without optimizations, we get ABI calling convention issues again (on the Rust side you pass something bad).
• brson: I can follow up with luqman today; I should see him. I'll make a note.
• jack: In theory, we could just merge it and tell people to build optimized for 32-bit.
• azita: Is there a list of stuff we need from Rust in Servo?
• [ed. note, there is now - High-pri/blockers: https://github.com/mozilla/servo/issues/2853 ; feature requests: https://github.com/mozilla/servo/issues/2854 ]

Stack sizes

• zwarich: Probably want to add stack space usage from Rust. Going to gather some numbers about how gigantic those frames are, b/c over 2mb stacks seem really large.
• kmc: There's a lint for that, probably.
• zwarich: Can't do it from just a lint because it's totally dependent upon what LLVM decides to do.
• kmc: Might want to consider a lint phase where we walk through the bitcode.
• zwarich: Can't figure out how much stack space until LLVM does register allocation.
• kmc: Aha! can't tell from the bitcode...
• zwarich: Fortunately, I think I can just write an awk script that can determine it from the dumped assembly.
• jack: It was amazing how many of the numbers were large on njn's numbers. We only have six tasks, and there were 5-6 things with giant stacks except for the fork/join concurrency pool.
• zwarich: If the green tasks are on one of many threads, you'd expect that a task that grew the stack significantly and got moved around would migrate to each native thread over time and expand the stack usage considerably.
• brson: I don't understand why these stacks claim to be > 2MB. Unless requested, Rustc will not create larger ones.
• jack: We must be asking for a giant stack somewhere in Servo, then! Based on zwarich and what you said, there must be something happening in rust-mozjs, and maybe it just travels through multiple threads.
• kmc: Might be leftover code from the segmented stack world.
• jack: Should follow up with jdm to see if it rings any bells for him.

graphics

• mrobinson: Wrote something that replaces the quad tree, but there are some OSX memory usage issues (not present on Linux). I think that thanks to zwarich's debugging, we know what's causing it and I'm writing a workaround now. If that works, we can just remove quadtree and then iterate on it over time. It's a very simple gridded tile implementation with uniform tiles. So I'm hoping to remove quadtree today or tomorrow.
• jack: zwarich, what was the memory usage?
• zwarich: From what I'm seeing, when it would scroll, it would ask for all tiles that it hadn't yet received again. If the render task is still rendering things you hadn't yet received, it would ask for them again. So you would allocate and render the same things again. A second problem is that when we want to free a buffer, the compositor has to ask the renderer, but it would be nice if the compositor owned the buffers itself. I will try to fix that, though I heard something from martin that maybe this decision was originally made for Android, so I'm just going to try to fix it and see what happens. And if anybody can rendering something other than a solid red page can check it out on Android... For GPU rendering, we'll have to make a skiagl context. I think I know what martin has to do to make it on parity with quadtree, but even the quadtree is going to have this problem. But it already has this problem - the render task is blocked on freeing buffers until it finishes current allocations/rendering.
• mrobinson: If the renderer owns it, it may be able to reuse buffers, though...
• zwarich: The compositor could certainly maintain a cache, and possibly more, since there is one render task per pipeline. We also currently don't do double-buffering, and once we do that we'll have to move it to the compositor anyway. I can't think of any reason for why the buffer management is in the render task.
• mrobinson: This problem uncovers a design issue. The layout tree change, the render tree realize that is has to render the contents, but the knowledge is split. In theory, though, the render task could have just sent the contents over. Right now, we have to ping-pong back and forth and I wonder if the latency of the messaging is causing this. There seems to be a lot of chattiness to get the contents of the layer.
• mbrubeck: The reason the buffers are handed back to the render thread is because of linux. There, the native layer object contains an X11 pixmap that has to be freed on the same thread where it's created. So, because of where it's currently allocated, we have to hand it back to be freed.
• mrobinson: Since pcwalton suggested replacing pixmaps with DRM on linux, we could just use shared memory on other platforms. I guess we already have that with CPU rendering mode, but shared memory may make more sense in that case, since you could just send the entire layer in one go instead of individual tiles, but there may be some tradeoffs there. I'm more used to the WebKit approach, where we just used shared memory, so I'd like to do some experimentation to see which is the faster approach.
• zwarich: On any platform with decent shared surfaces (OSX, Windows, Android/B2G), there's nothing lost by using tiles. And if you want efficient smooth scrolling, you want to recomposite prerendered buffers, which is the impetus for going to tiling on mobile platforms. If you really want to save memory, you can try not having both double-buffered tiles, but you'll lose responsiveness. It would be interesting to investigate both modes of rendering. But we should not move to directly rendering into...
• mrobinson: That would break accelerated rendering or video - we couldn't do it!
• zwarich: And if you're doing it on the GPU, you'd have to do a pre-blit pass on the surface, make a copy of the entire thing, etc. So the tile-based approach is better there too.
• mrobinson: Probably depends on GPU vs. CPU perf and the quality of the rasterizer.
• zwarich: We shouldn't let linux hold us down and just do something else on Linux. We should not architect around the limitations of desktop linux that cripple other platforms. Even Gecko isn't doing that anymore!

HTML parser

• jack: Status of HTML parser?
• kmc: Still various parts of tree building not implemented plus wiring it up to servo. It's probably 2-4 weeks, but coming along very well. I've implemented the in- cases, which is most of the work.

Continuous integration stuff

• jack: Building rust snapshots by hand
• larsberg: yes.
• jack: Still having weird failure?
• larsberg: I haven't taken a look at it yet; I was working on getting snapshots fixed and on Spidermonkey upgrade / Rust upgrade fixes. Are people still seeing the failures?
• jack: Yes, there was one this weekend I think.
• zwarich: The failure was the same as I saw last week, with the Linux builder just dying suddenly. This one was on an actual build, not a CI pre-build, but otherwise the same.
• larsberg: It's probably running out of some resource (memory, processes) and we just need to clamp down.
• zwarich: We wouldn't want to disable parallel layout on Travis since that finds problems. What about changing the -j argument to make? Or test parallelism?
• jack: Where does the failure happen? make check-content?
• zwarich: Yes, at least for the one I saw. It's annoying that there's no reliable way to test a fix.
• jack: Any update on paying Travis for increased resources and support?
• larsberg: Yes, they just got back to me; they can do all the needed work; just need to put together a statement of work.
• jack: Let's go ahead with that ASAP.
• cat: MEOW!