Dear A:
You wrote:
>I would like to know if it is possible to say :
>”Let`s come back to + location”
>In my opinion, the only way is “let`s go back to + location.”
>Example : “let`s go back to our office” and not “let`s come back to our office”

Normally, you would have to say “Go back” because you are talking about “us” (let us) and we are no longer at the office. The office is “back there” somewhere.

The only time it would make sense to say “let’s come back” would be if the speaker were at the office but the others were not. However, it would be unusual to say “let us come back”; the speaker in that case would be much more likely to simply say “come back” (the second person imperative).

“Come back to me” is a common line in songs, but that is spoken to someone who has gone away.