Why does it matter what my Echo does not know?

Because an Echo that knows everything is nobody. Yours is told the edge of what you could be sure about; past it, it answers the way you would — "no idea", "never been there", "that was my brother's thing". The refusals are what make it credible, far more than the answers.