example of applying CFR to chat_game

[YanSong97]

Hi,

I am trying to replicate the results presented in the paper States as Strings as Strategies: Steering Language Models with Game-Theoretic Solvers. I have read chat_game_psro_example.py which is a great demo. I was wondering if there is a similar example or demo applying CFR to the chat_game environment?

I have tried directly switching the environment in open_spiel/python/examples/cfr_example.py to chat_game but got the following error:

Traceback (most recent call last):
  File "/Desktop/code/open_spiel/ys/cfr_chat.py", line 178, in <module>
    cfr_solver = cfr.CFRSolver(game)
                 ^^^^^^^^^^^^^^^^^^^
  File "/Desktop/code/open_spiel/open_spiel/python/algorithms/cfr.py", line 500, in __init__
    super(CFRSolver, self).__init__(
  File "/Desktop/code/open_spiel/open_spiel/python/algorithms/cfr.py", line 170, in __init__
    self._initialize_info_state_nodes(self._root_node)
  File "/Desktop/code/open_spiel/open_spiel/python/algorithms/cfr.py", line 206, in _initialize_info_state_nodes
    index_in_tabular_policy=self._current_policy.state_lookup[info_state])
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'day_prefs:\nExample-0\nooo_days:\nExample-3\n\nFull Dialogue\n\n\n\n############################\nSchedule Proposal Message:\nfrom: Suzy\nto: Bob\n############################\n\n############################\nSchedule Proposal Message:\nfrom: Suzy\nto: Bob\n############################\n\n\nThat all sounds good to me.

It seems to be caused by a mismatch between TabularPolicy.state_lookup and info_state of the root node.

[lanctot]

Hi @YanSong97,

Good question! I will reach out to the original author and see if we can explain this and maybe provide one.

[lanctot]

Just an update: @imgemp and I are looking into it.

[imgemp]

I will put together an example, but two things I would think to check are:

1. I used cfr_solver = pyspiel.CFRSolver(game) in my implementation. I can't remember if I tried cfr.CFRSolver.
2. You would want to set provides_information_state...=True in the GAME_TYPE in in chat/chat_game.py.

[lanctot]

Ok, @imgemp made some changes and added a CFR example in bd204e5

@YanSong97: can you pull changes, and try this new example?

[YanSong97]

Yes, it works. Great thanks!