This paper introduces a reinforcement learning approach for optimal trade execution using both market and limit orders in a limit order book. The authors formulate the problem as a dynamic allocation task where the agent decides at each time step how to allocate its remaining inventory among market orders, limit orders at various price levels, and holding outside the book. Actions are modeled using a logistic-normal distribution over the simplex, which ensures valid allocations and allows for an efficient policy gradient derivation. The state space includes detailed market information (order book volumes, order flows, price drift) and private information (time, inventory, positions of active limit orders). The algorithm is trained in a simulated market environment with multiple agent types: noise traders submitting random orders, tactical traders reacting to order book imbalances, and a strategic trader (the algorithm itself). The simulation captures both direct market impact (from consuming liquidity) and indirect impact (from other agents reacting to the algorithm's actions). Numerical experiments demonstrate that the proposed reinforcement learning algorithm outperforms traditional benchmark strategies such as time-weighted average price (TWAP) and a submit-and-leave strategy, as well as an alternative reinforcement learning algorithm based on the Dirichlet distribution. The results highlight the importance of flexible order placement and cancellation, and the ability to adapt to market conditions in real time. The logistic-normal policy gradient method is general and can be applied to other dynamic allocation problems beyond trade execution.