Whisk together fish sauce, 1/4 cup vinegar, sugar, shallot, Sriracha, and ginger. Transfer tofu to a baking dish and top with marinade. Marinate at room temperature, 30 minutes.
Meanwhile, toss carrots with remaining vinegar. Season with salt and refrigerate.
Preheat a grill or grill pan to medium-high heat. Coat with vegetable oil and add tofu (reserve marinade). Grill, turning, until crisp and charred in spots, 2 to 3 minutes per side.
Generously brush cut sides of baguette with olive oil. Grill until toasted and golden, about 1 minute. Top bread with grilled tofu, carrots, herbs, and a drizzle of reserved marinade.
Marinated in Sriracha, tofu takes on a hint of sweet chile. Quick-pickled carrots add acidity and crunch.