fat muscle memory

i heard when you lose weight fat tissue remains. you only empty the fat tissue. does it mean you have fat memory too ? if you were overweight for a while does it mean you will gain weight much easier? if so dirtybulking is a very bad idea even for an ecto