Or-Parallelism has been implemented with quite limited overhead. The two major implementations proposed, Muse and Aurora, respectively report an average overhead over SICStus Prolog (on which both the implementations are based) of 8-22% (depending on the architecture considered) and 25%. These results are extremely encouraging.
The main advantage of or-parallelism (w.r.t. and-parallelism) is the fact that in or-parallelism it becomes quite natural to limit the overhead to the actual exploitation of parallelism. The compiler may detect (eventually through the help of user declarations) which are the choice points which may become parallel, and they will be appropriately marked (an almost zero-cost operation) when created. As long as parallelism is not exploited, choice points are treated normally, without any overhead.
Overheads generated in an or-parallel model depend on the specific model employed. Nevertheless, it is possible to make some general considerations. Gupta and Jayaraman  have observed that any or-parallel model implementing Prolog semantics is going to suffer a non-constant overhead in at least one of the following three phases: