In a small bowl combine yogurt, 3 tablespoons honey, and zest; set aside.
Using the large holes of a box grater, grate the ginger (no need to peel) into a small bowl. Squeeze ginger through a fine-meshed sieve or strainer placed over a medium bowl to get a total of 1 tablespoon ginger juice. Discard pulp.
To the bowl with the juice, add remaining 2 tablespoons honey and lemon juice; whisk to combine. Add papaya, blackberries, and raspberries and toss gently to coat.
To serve: Spoon half the fruit and juices among six 8-ounce tall glasses. Sprinkle chopped mint over the fruit. Top with half the yogurt mixture and half the granola. Layer with the remaining fruit, yogurt, and granola. Garnish with mint sprigs.