Unmanned Aerial Vehicles (UAVs) are becoming more popular than ever in outdoor commercial services. Many third-party UAV companies offer their UAVs as mobile data collectors to assist their clients in remote data collection missions. However, due to the lack of trust and transparency, the clients often have very little control on the behavior of these UAVs. This issue is even exacerbated if the service had to deal with private client data. In this work, we propose a solution that enables third-party UAVs to collect and process private client data from remote data sites in a trustworthy and efficient manner. We design and implement the Secure Homomorphic Encryption (SHE) framework. SHE combines trusted hardware enclave and homomorphic encryption technologies to provide strong privacy primitives on client data. SHE features in a recrypt technique such that the computation and communication overhead for homomorphic encryption on the client data is minimized. In addition, SHE takes the advantage of UAVs' travelling time to run data aggregation tasks in order to speed-up data processing. Through laboratory experiments, we demonstrate that SHE can meet the performance requirement in many common data processing and aggregation missions. SHE thus can be introduced as a trustworthy framework for the third-party UAV service providers.