The paper considers distributed control of human-in-the-loop multi-agent systems (MASs) using only relative input-output measurements. A human supervises the team through broadcasting a command signal to only one agent, called the leader, in response to any changes. To impose the human control signal, the leader needs to be a non-autonomous agent. All followers autonomously synchronize to the leader by locally interacting with each other over a communication graph and using only the relative input-output information. To decrease the communication burden, a novel distributed observer is presented for leader-follower MASs with non-autonomous leaders to approximate the relative state information of the agents. To obviate the requirement of knowing an upper bound on the human input, each agent adaptively estimates the human force by communicating its estimation with its neighbors. It is shown that the gain margin of the proposed distributed observer is infinity. The provided simulation results show effectiveness of the proposed method.