Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fktables for NLO theory #62

Closed
21 of 44 tasks
andreab1997 opened this issue May 24, 2022 · 43 comments
Closed
21 of 44 tasks

Fktables for NLO theory #62

andreab1997 opened this issue May 24, 2022 · 43 comments
Assignees

Comments

@andreab1997
Copy link
Contributor

andreab1997 commented May 24, 2022

This is the NLO eqiuivalent of NNPDF/fktables#5

We list here the Fktables which we cannot compute at the moment for the central NLO theory (208):

  • jets (so needs Composite Output eko#105)
    • ATLAS_2JET_7TEV_R06
    • ATLAS_1JET_8TEV_R06
    • CMS_1JET_8TEV
    • CMS_2JET_7TEV
  • missing grids
    • ATLAS_WCHARM_WP_DIFF_7TEV (take from dom /media/Fk/fktables/data/appl_subgrids)
    • ATLAS_WCHARM_WM_DIFF_7TEV (take from dom /media/Fk/fktables/data/appl_subgrids)
    • CMSWCHARMTOT (Write the yamldb for the composition CMSWCHARM_WP and CMSWCHARM_WM with operand "add", take the two grids from /media/Fk/fktables/data/appl_subgrids )
    • CMSWCHARMRAT (same of previous but with operands "ratio")
    • CMS_WCHARM_DIFF_UNNORM_13TEV (same as the previous but with operand "add" and different grids: CMS_WCHARM_13TEV_WMC and CMS_WCHARM_13TEV_WPCB)
  • We miss ren sv:
    • ATLAS_TOPDIFF_DILEPT_8TEV_TTRAPNORM (because we are missing ren sv for CMSTTBARTOT8TEV-TOPDIFF8TEVTOT, coming from Sherpa)
    • ATLAS_TTBARTOT_13TEV_FULLLUMI (because we are missing ren sv for ATLAS_TTBARTOT_13TEV_FULLLUMI-TOPDIFF13TEVTOT, coming from Sherpa)
    • ATLASTTBARTOT7TEV (because of ATLASTTBARTOT7TEV-TOPDIFF7TEVTOT, coming from Sherpa)
    • ATLASTTBARTOT8TEV (because of ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT, this should be the same as CMSTTBARTOT8TEV-TOPDIFF8TEVTOT)
    • ATLAS_WM_JET_8TEV_PT ( because of ATLAS_WM_JET_8TEV_PT-atlas-atlas-wjets-arxiv-1711.03296-xsec003 which is coming from ploughshare)
    • ATLAS_WP_JET_8TEV_PT ( because of ATLAS_WP_JET_8TEV_PT-atlas-atlas-wjets-arxiv-1711.03296-xsec002 which is coming from ploughshare)
    • ATLASZPT8TEVMDIST (it is coming from MCFM)
    • ATLASZPT8TEVYDIST (it is coming from MCFM)
    • CMS_1JET_8TEV (coming from NLOjet++)
    • CMSTOPDIFF8TEVTTRAPNORM (because of CMSTOPDIFF8TEVTTRAPNORM-TOPDIFF8TEVTTRAP, coming from Sherpa and total cross-section see above)
    • CMSTTBARTOT13TEV (because of CMSTTBARTOT13TEV-TOPDIFF13TEVTOT, coming from Sherpa)
    • CMSTTBARTOT8TEV (because of CMSTTBARTOT8TEV-TOPDIFF8TEVTOT, coming from Sherpa)
    • CMSTTBARTOT7TEV (because of CMSTTBARTOT7TEV-TOPDIFF7TEVTOT, coming from Sherpa)
    • CMSZDIFF12 (coming from MCFM)
    • ATLAS_WCHARM_WP_DIFF_7TEV
    • ATLAS_WCHARM_WM_DIFF_7TEV
    • CMSWCHARMTOT
    • CMSWCHARMRAT
    • CMS_WCHARM_DIFF_UNNORM_13TEV
  • FTDY (needs VRAP)
    • DYE886R_dw_ite
    • DYE886P
    • DYE906R_dw_ite
  • Things to check:
    • D0ZRAP_40 (working for NNLO, check the bug)
    • D0WMASY
    • ATLASWZRAP36PB
    • ATLASDY2D8TEV
    • ATLAS_DY_2D_8TEV_LOWMASS
    • ATLASPHT15_SF (two bins for each FKtable are correct the others are not)
    • CMSWEASY840PB
    • CMSWMASY47FB
    • CMSWMU8TEV
  • Things with a conjectured solution
    • ATLAS_TTB_DIFF_8TEV_LJ_TRAPNORM (maybe just a 2.0 conversion factor)
    • ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM (same)
    • CDFZRAP_NEW (see issue NNPDF/fktables#17)
    • CMS_TTBAR_2D_DIFF_MTT_TRAP_NORM (this is actually correct NNPDF/fktables#5 )
@felixhekhorn felixhekhorn changed the title Missing Fktables for NLO theory Fktables for NLO theory May 24, 2022
@andreab1997
Copy link
Contributor Author

@cschwan @enocera Could you please have a look to the grids with the missing ren scale variations of the list above (third list)? How should we proceed?

For example we are missing lots of total cross sections for TTBAR which are not in the pinecard repo. However the repo provides differential cross-sections. Should we add the missing?

@cschwan
Copy link
Contributor

cschwan commented Sep 26, 2022

The top-pair production grids should be available in https://github.com/NNPDF/pineapplgrids, one of them possibly in a branch. All other would have to be implemented in https://github.com/NNPDF/runcards.

@andreab1997
Copy link
Contributor Author

The top-pair production grids should be available in https://github.com/NNPDF/pineapplgrids, one of them possibly in a branch. All other would have to be implemented in https://github.com/NNPDF/runcards.

Thank you very much. So the grids in https://github.com/NNPDF/pineapplgrids contain ren scale variations?

@felixhekhorn
Copy link
Contributor

@cschwan for the elements in the fifth list above ("things to check") we don't have any clear clue on what the problem is ... some of the bins (but not all) are matching and others are completely off. Do you see any common pattern for the datasets? @andreab1997 checked that the ekos are fine and also the grids are fine (as we knew from before). So now we're suspecting maybe pineko or pineappl (i.e. Grid::convolute_eko) to have a breaking change somewhere (tentatively between May and now). We concluded this from a bunch of files at dom for which instead the convolution was working (and which got computed before that date). E.g. NNPDF/pineappl#103 would fall into this time, do you think there is a possibility that this broke the convolution?

As for pineko: there is actually almost nothing, but the Grid::convolute_eko call, so while it is still possible the bug might be there, I'd say it is unlikely ...

@cschwan
Copy link
Contributor

cschwan commented Sep 27, 2022

Thank you very much. So the grids in https://github.com/NNPDF/pineapplgrids contain ren scale variations?

They should! You can explicitly check this if you run the CLI using pineappl obl --orders [grid] on them, and this should show the logmur and logmuf orders. Let me know if there are any problems!

@cschwan
Copy link
Contributor

cschwan commented Sep 27, 2022

[..] We concluded this from a bunch of files at dom for which instead the convolution was working (and which got computed before that date). E.g. N3PDF/pineappl#103 would fall into this time, do you think there is a possibility that this broke the convolution?

This could well be, yet another reason to have a unit test for this ...

I can test this, but I'm afraid I won't be able to this week. If you want to test this yourself, try

git checkout 954aadfbf3878d0f0583c820b3f5326462a12478

which will checkout PineAPPL before the changes. I hope that the Python API still works.

@andreab1997
Copy link
Contributor Author

[..] We concluded this from a bunch of files at dom for which instead the convolution was working (and which got computed before that date). E.g. N3PDF/pineappl#103 would fall into this time, do you think there is a possibility that this broke the convolution?

This could well be, yet another reason to have a unit test for this ...

I can test this, but I'm afraid I won't be able to this week. If you want to test this yourself, try

git checkout 954aadfbf3878d0f0583c820b3f5326462a12478

which will checkout PineAPPL before the changes. I hope that the Python API still works.

I have done some tests and I can add some details. As expected, I was able to produce the correct Fktables with older versions of pineko, eko and pineappl. In particular I used pineko==0.1.1, eko==0.8.5 and pineappl==0.5.2. @felixhekhorn @cschwan Do this suggest something to you?
Take into account that with pineko==0.1.2 and pineappl==0.5.4 I got again the wrong results.

@alecandido
Copy link
Member

You should be able to do the same with every version of eko. Please, try the latest one.

@andreab1997
Copy link
Contributor Author

You should be able to do the same with every version of eko. Please, try the latest one.

Yes I can try but pineko==0.1.1 requires eko==0.9.0 at most. Anyway, It is not a problem most likely.

@alecandido
Copy link
Member

Apart from what the package require, you should be able to use a newer version (if not through Poetry, manually).

Might be that something is broken, but something is not everything. Or do you know that there is something required in pineko==0.1.1 that is not any longer there in eko==0.10.x?

@andreab1997
Copy link
Contributor Author

Apart from what the package require, you should be able to use a newer version (if not through Poetry, manually).

Might be that something is broken, but something is not everything. Or do you know that there is something required in pineko==0.1.1 that is not any longer there in eko==0.10.x?

I don't know, most likely nothing is broken. Indeed I launched the job and it did not crash until now. It was just a possibility :)

@andreab1997
Copy link
Contributor Author

Apart from what the package require, you should be able to use a newer version (if not through Poetry, manually).

Might be that something is broken, but something is not everything. Or do you know that there is something required in pineko==0.1.1 that is not any longer there in eko==0.10.x?

I can confirm that even with eko==0.10.2, with the older versions of pineko and pineappl, I can obtain the correct results.

@cschwan
Copy link
Contributor

cschwan commented Sep 28, 2022

So do we conclude that Grid::convolute_eko is broken?

@alecandido
Copy link
Member

For the tested data set, yes.

Unless we are deeply misunderstanding something.

@cschwan
Copy link
Contributor

cschwan commented Sep 28, 2022

OK; is there an immediate action needed or is rolling back to a previous version sufficient?

@andreab1997
Copy link
Contributor Author

OK; is there an immediate action needed or is rolling back to a previous version sufficient?

It is sufficient to get the correct results. However, since we need to downgrade even pineko, we have other modifications that we probably don't want. Moreover, it is a lot slower (factor 6 more or less) which is a problem given the number of fktables to compute. So I believe that we need to solve the bug.

@cschwan
Copy link
Contributor

cschwan commented Sep 28, 2022

We definitely need to fix this. However, I'm a bit busy for the remainder of this week and therefore I was trying to gauge the urgency; from your answer I suppose it is urgent, so we have to come up with a plan. Do you have an idea in which cases the generation fails (all processes or just some)? Ideally we'd implement a minimal test case here: NNPDF/pineappl#122. Does the generation of the positivity FK tables fail?

@andreab1997
Copy link
Contributor Author

We definitely need to fix this. However, I'm a bit busy for the remainder of this week and therefore I was trying to gauge the urgency; from your answer I suppose it is urgent, so we have to come up with a plan. Do you have an idea in which cases the generation fails (all processes or just some)? Ideally we'd implement a minimal test case here: N3PDF/pineappl#122. Does the generation of the positivity FK tables fail?

I don't think that the positivity fails, but I did not test recently. If you want I can. The list of the datasets that are failing is the fourth ("Thing to check"), while for example all the drell yan are not (and of course there are others that are correct).

@alecandido
Copy link
Member

Just to point out that @felixhekhorn start doubting the problem being PineAPPL in another thread NNPDF/pineappl#103 (comment)

@andreab1997
Copy link
Contributor Author

[..] We concluded this from a bunch of files at dom for which instead the convolution was working (and which got computed before that date). E.g. N3PDF/pineappl#103 would fall into this time, do you think there is a possibility that this broke the convolution?

This could well be, yet another reason to have a unit test for this ...

I can test this, but I'm afraid I won't be able to this week. If you want to test this yourself, try

git checkout 954aadfbf3878d0f0583c820b3f5326462a12478

which will checkout PineAPPL before the changes. I hope that the Python API still works.

I have tried to use the version of pineappl given by this checkout together again with pineko=0.1.1 and it worked for the dataset I am testing (which is ATLASWZRAP36PB). Of course it would be nice to instead use pineko=0.1.2 but this is not doable unfortunately because it is uncompatible with that version of pineappl. So at the end we can only conclude that the error, if it is in pineappl, is after this commit and, if it is in pineko, is between 0.1.1 and 0.1.2.

@andreab1997
Copy link
Contributor Author

[..] We concluded this from a bunch of files at dom for which instead the convolution was working (and which got computed before that date). E.g. N3PDF/pineappl#103 would fall into this time, do you think there is a possibility that this broke the convolution?

This could well be, yet another reason to have a unit test for this ...
I can test this, but I'm afraid I won't be able to this week. If you want to test this yourself, try

git checkout 954aadfbf3878d0f0583c820b3f5326462a12478

which will checkout PineAPPL before the changes. I hope that the Python API still works.

I have tried to use the version of pineappl given by this checkout together again with pineko=0.1.1 and it worked for the dataset I am testing (which is ATLASWZRAP36PB). Of course it would be nice to instead use pineko=0.1.2 but this is not doable unfortunately because it is uncompatible with that version of pineappl. So at the end we can only conclude that the error, if it is in pineappl, is after this commit and, if it is in pineko, is between 0.1.1 and 0.1.2.

Of course I am assuming that if it is working for ATLASWZRAP36PB (that was wrong with the most recent versions) it would work for all the others that were wrong. I believe it is a very reasonable assumption but we should keep this in mind while debugging. Of course you have any other suggestion for datasets to test, please tell me. We have chosen ATLASWZRAP36PB only because is reasonably little and fast.

@cschwan
Copy link
Contributor

cschwan commented Sep 29, 2022

Do you have the EKO for ATLASWZRAP36PB as a file somewhere where I have access to (dom, montblanc)?

@andreab1997
Copy link
Contributor Author

Do you have the EKO for ATLASWZRAP36PB as a file somewhere where I have access to (dom, montblanc)?

Yes it is on dom. Precisely in /media/FK/fktables_pineko/data/ekos/208/. Of course there are three ekos there because the dataset is actually composed by three FKs. I am actually testing the Z0 one.

@cschwan
Copy link
Contributor

cschwan commented Sep 29, 2022

Great, thanks a lot - that's exactly what I need. I'll try to build a unit test for Grid::convolute_eko with that and then we'll see if and where there's a problem!

@andreab1997
Copy link
Contributor Author

Great, thanks a lot - that's exactly what I need. I'll try to build a unit test for Grid::convolute_eko with that and then we'll see if and where there's a problem!

Perfect, thank you very much.

@andreab1997
Copy link
Contributor Author

The top-pair production grids should be available in https://github.com/NNPDF/pineapplgrids, one of them possibly in a branch. All other would have to be implemented in https://github.com/NNPDF/runcards.

Looking at the grids that are in https://github.com/NNPDF/pineapplgrids, none of them are what we really need (as specified in the list "We miss ren sv" above. I was wondering if maybe they have just different names but then, is there a "dictionary" somewhere?

@alecandido
Copy link
Member

The dictionary at the moment is inside yamldb, I'd say. Eventually, it will be inside the new commondata.

Essentially, the two nomenclatures are:

  • the original NNPDF one (rather confused, we deprecated)
  • the new one by @cschwan, that is the one inside that repo

Unfortunately, if they were not in theory 200/400, you have to find the mapping on your own. However, it is not unlikely that quite a few grids have still to be generated.

@andreab1997
Copy link
Contributor Author

The dictionary at the moment is inside yamldb, I'd say. Eventually, it will be inside the new commondata.

Essentially, the two nomenclatures are:

  • the original NNPDF one (rather confused, we deprecated)
  • the new one by @cschwan, that is the one inside that repo

Unfortunately, if they were not in theory 200/400, you have to find the mapping on your own. However, it is not unlikely that quite a few grids have still to be generated.

Yes but in fact the names that I have put in that list are exactly coming from the yamldb of theory 400. Problem is that the grids that are listed there are not in that repo. So is there a new version (with respect to the theory 400) of the yamldb somewhere?

@andreab1997
Copy link
Contributor Author

The dictionary at the moment is inside yamldb, I'd say. Eventually, it will be inside the new commondata.
Essentially, the two nomenclatures are:

  • the original NNPDF one (rather confused, we deprecated)
  • the new one by @cschwan, that is the one inside that repo

Unfortunately, if they were not in theory 200/400, you have to find the mapping on your own. However, it is not unlikely that quite a few grids have still to be generated.

Yes but in fact the names that I have put in that list are exactly coming from the yamldb of theory 400. Problem is that the grids that are listed there are not in that repo. So is there a new version (with respect to the theory 400) of the yamldb somewhere?

For example, for ATLAS_TOPDIFF_DILEPT_8TEV_TTRAPNORM the yaml file looks for ATLAS_TTB_8TEV_2L_TTRAP and
CMSTTBARTOT8TEV-TOPDIFF8TEVTOT. The former I already have but the latter is not in that repo. The same applies for all the others it seems.

@cschwan
Copy link
Contributor

cschwan commented Oct 7, 2022

All the *-{7,8,13}TEVTOV grid should be those:

However, it's possible that the scales are chosen differently in those grids, so if you compare against the APPLgrid-converted grids keep that in mind.

@andreab1997
Copy link
Contributor Author

All the *-{7,8,13}TEVTOV grid should be those:

However, it's possible that the scales are chosen differently in those grids, so if you compare against the APPLgrid-converted grids keep that in mind.

Ok, thank you! So, for example, for ATLASTTBARTOT7TEV I should use ATLAS_TTB_7TEV_TOT.pineappl.lz4 even if in the yaml file I have ATLASTTBARTOT7TEV-TOPDIFF7TEVTOT.pineappl.lz4, right? (i.e. just the names are changed)

@cschwan
Copy link
Contributor

cschwan commented Oct 10, 2022

As far as I know they're all the same grid, although they have different names. You should check that, for instance with the CLI: pineappl diff [GRID1] [GRID2] [PDFSET]. If they're not the same let me know!

@andreab1997
Copy link
Contributor Author

andreab1997 commented Oct 10, 2022

As far as I know they're all the same grid, although they have different names. You should check that, for instance with the CLI: pineappl diff [GRID1] [GRID2] [PDFSET]. If they're not the same let me know!

Actually I tried but it results in an error.

pineappl diff ../data/grids/4/ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4 pineapplgrids/ATLAS_TTB_8TEV_TOT.pineappl.lz4 NNPDF40_nlo_as_01180
Error: unable to read 'pineapplgrids/ATLAS_TTB_8TEV_TOT.pineappl.lz4'

Caused by:
unknown array version: 118

Am I doing something wrong?

@cschwan
Copy link
Contributor

cschwan commented Oct 10, 2022

Did you check out the pineapplgrids repository with LSF support? It could be that they are simply text files if you didn't check them out properly. But in any case, you should compare the various grids converted from the APPLgrids with each other, if you compare the pinefarm-generated grids with the APPLgrid-converted grids you'll probably see 5-10% changes.

@andreab1997
Copy link
Contributor Author

andreab1997 commented Oct 11, 2022

Did you check out the pineapplgrids repository with LSF support? It could be that they are simply text files if you didn't check them out properly. But in any case, you should compare the various grids converted from the APPLgrids with each other, if you compare the pinefarm-generated grids with the APPLgrid-converted grids you'll probably see 5-10% changes.

Yes this was the problem, thank you! However if now I try pineappl diff ../data/grids/4/ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4 pineapplgrids/ATLAS_TTB_8TEV_TOT.pineappl.lz4 NNPDF40_nlo_as_01180 I get Error: selected orders differ. I checked the orders separately and indeed it seems that ATLAS_TTB_8TEV_TOT.pineappl.lz4 contains QED corrections while ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4 does not. Of course if I compare the common orders (so the as^2 and as^3) they are matching almost perfectly.

So is this enough to conclude that I can use ATLAS_TTB_8TEV_TOT.pineappl.lz4 in place of the ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4? (and the same for the others I guess)

@felixhekhorn
Copy link
Contributor

If you do pineappl diff --help you can see there is a handy --ignore-orders and most likely you also need --ignore-lumis

@andreab1997
Copy link
Contributor Author

However I am confused by the x1 entry that I get using pineappl orders because for ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4 I get 0 1 while for ATLAS_TTB_8TEV_TOT.pineappl.lz4 I get 0.5 1.5. Is this correct?

@felixhekhorn
Copy link
Contributor

This should be a fully inclusive cross section so any binning (x1) is just fake - however, pineappl requires some sort of binning for technical reason and so you can create whatever bin ... it should not matter

@alecandido
Copy link
Member

So is this enough to conclude that I can use ATLAS_TTB_8TEV_TOT.pineappl.lz4 in place of the ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4? (and the same for the others I guess)

If you can, provide a yamldb entry spelling the actual translation.

The yamldb has been initially generated from generate.py, making use of apfelcomb.db (check NNPDF/fktables#15). So, by default it was making use of all old names (those that APFELcomb was aware of).
However, part of them were translated to the new nomenclature:
https://github.com/NNPDF/fktables/blob/e36d2ce5351105fbe69272b58d79f54692493b0a/data/fktocommon/translate.py#L34-L39

But you see it was a manual action, based on @cschwan explaining part of the matching, and @felixhekhorn finding the rest of them. If you are lucky, you can find them all there (but then most likely also in the yamldb), otherwise you have to find the matching yourself.

So, yes, if they match (common orders) they are the same.

@alecandido
Copy link
Member

Maybe we should prompt to speed up new Commondata implementation, since they will be the proper and final solution.

@felixhekhorn
Copy link
Contributor

Maybe we should prompt to speed up new Commondata implementation, since they will be the proper and final solution.

this is a nice to have and the proper solution, but we are not correlated to them at the moment - we're maintaining our own DB

@alecandido
Copy link
Member

this is a nice to have and the proper solution, but we are not correlated to them at the moment - we're maintaining our own DB

But our own DB has no proper source. Until now we use the instance inside theory 400, and now is not enough any longer.

@alecandido alecandido transferred this issue from another repository Nov 30, 2022
@cschwan
Copy link
Contributor

cschwan commented Dec 8, 2022

The renormalization-scale grids:

  • all top-pair production named CMSTTBARTOT8TEV-TOPDIFF8TEVTOT should be total cross sections and therefore called ATLAS_TTB_{7,8,13}TEV_TOT. If that's the case the scales are chosen dynamically (opposed to statically mur = muf = Mt in the old grids), so you might see differences. But I have to check to confirm all that.
  • for ATLAS_WM_JET_8TEV_PT and ATLAS_WP_JET_8TEV_PT we have Add ATLAS_WJET_8TEV_PTW datasets pinecards#100, so there are at least runcards, but I never uploaded the grids because the comparison with the old ones failed (see the linked Issue).
  • for ATLASZPT8TEVMDIST and ATLASZPT8TEVYDIST we have Add ATLASZPT8TEV{M,Y}DIST datasets pinecards#61, similar story as above.
  • for jets we have runcards and it should be possible to run them for at least NLO QCD. I always tried with NLO QCD + NLO EW/mixed, but that's probably not a good idea. For this we should be able to generate grids!
  • CMSZDIFF12 I never implemented, but we should be able to generate predictions.
  • W+charm is probably not possible at NLO without more work. But they're not needed for NNLO QCD.
  • for jets we now have NNLO QCD grids, and even better we have the fastNLO flexible scale grids (see below). These should contain all renormalization/factorization logs subgrids (I have to check though).

Jet grids on ploughshare:

We can convert those now, however the conversion of the log grids isn't implemented yet, see NNPDF/pineappl#171.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants