Mobile edge computing (MEC) is an evolutionary architecture integrating mobile access and edge computing. MEC can not only enhance user experience by improving the latency, but also save energy by computing tasks offloaded partially or fully from user equipments (UEs) to the MEC server. In this paper, we consider an MEC network with multiple UEs and multiple base stations (BSs) which are all equipped with MEC servers. All UEs can offload their computing tasks to one of the BSs via uplink transmissions. Both conventional optimization methods and learning-based approaches are proposed to optimize the UE data offloading and MEC computational resource allocation under latency constraints with the aim to minimize the global energy consumption. MEC selection is taken into account in the learning based approach. We adopt actor-critic (AC) reinforcement learning in the learning based algorithm, which is used to decide first the MEC selection and subsequently the offloading data ratio. Simulation results demonstrate that our proposed learning based algorithm can address all the considered scenarios and lead to performance improvements compared to baseline strategies.