How to plot QuadrantGates?

[hrj21]

Hi all, thanks for making such an excellent package. My question is: what's the correct way to visualize quadrant gates? I tried applying the same method I use for rectangle, polygon, and ellipsoid gates, but I can't get it to work. Here is an MWE (assuming test_data_diamond_01.fcs is in data in the working directory):

# import packages
import bokeh
from bokeh.plotting import show
import flowkit as fk

# create session and add siamond sample
session = fk.Session()
session.add_samples('data/test_data_diamond_01.fcs')

session.get_sample('test_data_diamond_01.fcs').channels

# define QuadrantDividers
quad_div1 = fk.QuadrantDivider(
    'chan-a-div',
    'channel_A',
    compensation_ref = None,
    values = 5e4
)

quad_div2 = fk.QuadrantDivider(
    'chan-b-div',
    'channel_B',
    compensation_ref = None,
    values = 5e4
)

quad_divs = [quad_div1, quad_div2]

# define Quadrants
quad_1 = fk.gates.Quadrant(
    quadrant_id    = 'double_pos',
    divider_refs   = ['chan-a-div', 'chan-b-div'],
    divider_ranges = [(5e4, None), (5e4, None)]
)

quad_2 = fk.gates.Quadrant(
    quadrant_id    = 'a_pos',
    divider_refs   = ['chan-a-div', 'chan-b-div'],
    divider_ranges = [(5e4, None), (None, 5e4)]
)

quad_3 = fk.gates.Quadrant(
    quadrant_id    = 'b_pos',
    divider_refs   = ['chan-a-div', 'chan-b-div'],
    divider_ranges = [(None, 5e4), (5e4, None)]
)

quad_4 = fk.gates.Quadrant(
    quadrant_id    = 'double_neg',
    divider_refs   = ['chan-a-div', 'chan-b-div'],
    divider_ranges = [(None, 5e4), (None, 5e4)]
)

quadrants = [quad_1, quad_2, quad_3, quad_4]

# define QuadrantGate
quad_gate = fk.gates.QuadrantGate(
    'a_b_quad',
    dividers  = quad_divs,
    quadrants = quadrants
)

session.add_gate(quad_gate, gate_path = ('root',))

# attempt plotting gates, each gives a different error
session.plot_gate('test_data_diamond_01.fcs', 'a_b_quad') 
session.plot_gate('test_data_diamond_01.fcs', 'a_pos') 
session.plot_gate('test_data_diamond_01.fcs', 'a_pos', gate_path = ('root',))

I came across #41 but couldn't find any documentation of the best way to do it. Ideally I would like to either plot all quadrants together in the same plot.

Best wishes
Hefin

EDIT: I also find I get a QuadrantReferenceError when running session.export_wsp(...), which I think may be related. So now I'm wondering if I've constructed the gate incorrectly.

[whitews]

Hi Hefin,

You're almost there. The issue is that the values argument for the QuadrantDivider class takes a list and not a float value. Changing those divider definitions to the following works for me:

# define QuadrantDividers
quad_div1 = fk.QuadrantDivider(
    'chan-a-div',
    'channel_A',
    compensation_ref=None,
    values=[5e4]
)

quad_div2 = fk.QuadrantDivider(
    'chan-b-div',
    'channel_B',
    compensation_ref=None,
    values=[5e4]
)

Note that plotting quadrant gates only works for the QuadrantGate ID reference and not for the individual quadrants, so your last 2 lines won't work. Also, before plotting gates you should run the analyze_samples method, though it will work without doing that in this case.

I do realize it is tedious to construct Quadrant gates. The current API is a mapping from the GatingML specification and I'd love to find a simpler way to create them.

Let me know if this helps,
Scott

[hrj21]

Thank you so much! How silly of me, yes that works perfectly now thank you. However, I still get

QuadrantReferenceError: cd4/8_double_pos references a Quadrant, specify the owning QuadrantGate GateNode('/root/scatter/singlets/cd3/cd4_cd8_quad', custom_gates={}, gate=QuadrantGate(cd4_cd8_quad, quadrants: 4), gate_type='QuadrantGate') instead

when I run

session.export_wsp('../data/gating_strategy.wsp', group_name = 'All samples')

Coming from R, the construction of the quadrants does feel like it has more steps than I would expect, but completely appreciate the importance of being GatingML compliant and I don't understand the complexities of what's going on under the hood. In case it's useful to compare the APIs, I've pasted an example below of how I'd draw and visualize quadrant gates in R using flowCore, flowWorkspace, and ggcyto. I would have assumed flowWorkspace is GatingML compliant but I could be wrong!

# load packages
library(flowCore) # for basic io and fcs file functions
library(flowWorkspace) # for GatingSet class
library(ggcyto) # for plotting

# read single example fcs
dat <- read.flowSet(system.file(
  "extdata",
  "0877408774.B08",
  package = "flowCore"
  )
)

# create GatingSet
gs <- GatingSet(dat)

## create example quadGate
qg <- quadGate(filterId = "myQuadGate1", "FSC-H" = 100, "SSC-H" = 400)

# add gate to GatingSet
gs_pop_add(gs, qg)
recompute(gs)

# plot result
plot(gs)
autoplot(
  gs, 
  gate = c("FSC-H-SSC-H+", "FSC-H+SSC-H-", "FSC-H+SSC-H+", "FSC-H-SSC-H-"), 
  bins = 150
)

Best wishes
Hefin

[whitews]

Hi Hefin,

Unfortunately, exporting Quadrant gates to a FlowJo workspace isn't supported yet. Attempting to do so was supposed to throw aNotImplementedError, however the error you reported is triggered earlier when collecting the gates to export.

The reason why it isn't implemented actually relates directly to the discussion of why the quadrant gates are so tedious to create. The GatingML specification overloads the definition of a quadrant gate beyond the true mathematical definition of 2 dividers creating 4 quadrants. It allows an arbitrary number of dividers along with merging of the resulting "quadrants". If you'd like to read it (great for insomnia!), the QuadrantGate definition is in section 5.4 (GatingML PDF). Here's an example they use that illustrates this:

The GatingML spec overloads things in a few other places as well. IMO, it is overly ambitious, trying to cover as many use cases as possible, but sacrificing simplicity. One could achieve the same result using range gates with Boolean gates. In any case, I believe this is why FlowJo (and other packages) do not implement GatingML compliant quadrant gates, and is also the reason FlowKit doesn't export the current QuadrantGate to FlowJo workspaces.

So, how to solve this? FlowKit does have "WSP" prefixed gates (actually just WSPEllipsoidGate for now) for FlowJo compatible versions of gates. I could create a simple WSPQuadrantGate that could hopefully match the simplicity of the flowWorkspace API. Curious your thoughts on whether that would be a good idea or just make things more confusing.

I should note the upcoming 1.2 release is focused on simplifying the API in situations like this. Another example I'd like to address is the admittedly odd way the rectangle / range boundaries are specified in the dimension instances instead of in the gate class directly (another artifact of mirroring GatingML). I'm not sure when it will be released but I would consider trying to work in a quadrant gate fix. And I welcome any other feedback you might have regarding the API.

Kind regards,
Scott

[hrj21]

Thanks for taking the time to give such a detailed answer. I hadn't realise that FlowJo's quadrant gates weren't GatingML compliant. I think if the reasons and use of WSP-prefixed gating methods is well-documented then I don't see an issue. Perhaps Flowkit could include a "quadrant" gating method that was simpler to use but was actually just syntactic sugar for defining 4 RectangleGates that meet at their inner vertex? Then there could be a separate method for creating truly GML-compliant quads if required. Alternatively, the export_wsp() method could just convert QuadrantGates into rectangles so they can be interpreted correctly by FlowJo (again, with this behaviour documented as such).

Anyway, I'm thoroughly enjoying Flowkit and this is a very minor thing.