For a good to be a giffen good, the positive income effect should outweigh negative substitution effect to actually violate law of demand. Hence that is the reason we refer to all giffen goods as inferior good but not all inferior goods are giffen.
The diagram below explains this
