Questions on MTSAC

[NagisaZj]

Description

In the documentation, the example commands for MTSAC-MetaWorld do not seem to include task index as model's input, which means that the model has no way of identifying different tasks.

How to reproduce

Add this line to the encode() function in mtrl/agent/components/actor.py:

print(self.should_concatenate_task_info_with_encoder)

Then run the provided example commands for MTSAC-MetaWorld:

PYTHONPATH=. python3 -u main.py \
setup=metaworld \
env=metaworld-mt10 \
agent=state_sac \
experiment.num_eval_episodes=1 \
experiment.num_train_steps=2000000 \
setup.seed=1 \
replay_buffer.batch_size=1280 \
agent.multitask.num_envs=10 \
agent.multitask.should_use_disentangled_alpha=True \
agent.encoder.type_to_select=identity \
agent.multitask.should_use_multi_head_policy=False \
agent.multitask.actor_cfg.should_condition_model_on_task_info=False \
agent.multitask.actor_cfg.should_condition_encoder_on_task_info=True \
agent.multitask.actor_cfg.should_concatenate_task_info_with_encoder=True

Stack trace / error message
It gives False.

Should I add agent.multitask.should_use_task_encoder=True if I want to concat task id with state? Or can you provide example codes for running MTSAC with task id concated with state?

[shagunsodhani]

Hi! The MTSAC uses different entropy regularizers for each task and hence task id is not passed to the encoder. I dont think we support concatenating task id. The closest thing is concatenating the task emebdding. For that, you can set agent.multitask.should_use_task_encoder=True