Workers fear returning to work. Many are resisting the call