Disclaimer: I did not use SWI-Prolog in a commercial/production environment, I did not try the SWI/Java interface (although it was reported on the SWI-Prolog mailing list to work reasonably well), and I have no interest in the semantic web.
That said, I did write a large and non-trivial project using SWI-Prolog, and it wasn't a toy application - I needed it to work in order to finish my research. Therefore I have some credentials for answering this question, even if not all of the required ones.
In the context of a large project, it is best to treat Prolog as a domain-specific language for writing search routines - Not unlike SQL and storage. Prolog does one thing extremely well, and that it solving search problems. If your problem can be narrowed down to finding a valid assignment to a set of variables given a set of facts and rules, Prolog is your weapon-of-choice: You won't have to worry about anything else in your coding other than tweaking the queries and rules. It will also run reasonably fast, especially if you compile the predicates. Prolog is often slow not because it is inherently slow, but because it runs search routines over a large search space.
Keeping that in mind, you can integrate it with Java in a similar manner you integrate SQL: Prolog takes care about solving the search. It stores the permanent data and the rules, and run the queries. The Java app takes care of anything else.
Development environments are a bit of a soft-spot for Prolog. You won't get a fancy IDE - Simply because there are very few things to be fancy about. Prolog programs rarely have full compile-time information, so you won't get perfect auto-completion. I simply used Emacs, with the Prolog shell always open. For most things, the Prolog textual tracer/debugger will do the trick. SWI comes with its-own graphical debugger, but I haven't tried it.
Prolog has several rarely-known advantages: A built-in parsing framework (DCG rules), a great macros system (term expansion), and meta-predicates (call, findall, etc.). If you know how to use them, these are real aces in your deck.
To summarize: Does Prolog boosts or damages productivity? That depends on what currently blocks your productivity. If you waste too much time wondering how to implement knowledge representation and search routines in Java - Just use Prolog; It's not as if you implement your-own red-black trees instead of using SQL. If you're looking for soft-realtime performance and a spoiling IDE - Try something else.
Great, this is the kind of answer I was looking for. The problem I have is to do with knowledge represented in form of a model (semantic owl model, but don bother too much with detail) and existing library written in prolog that could represent this knowledge and expose this knowledge for searching and inference capabilities. ( I am going to break comment to maintain a semblance of coherency)
Contd: The part that, you could explain further is this statement "Keeping that in mind, you can integrate it with Java in a similar manner you integrate SQL". How can I pass data to prolog and get the result back into java.?
@doc_180: You have at least 3 simple options. (a) Run the SWI-Prolog HTTP server and use it as a REST web service. (b) Open a pipe to the Prolog process and send queries as text and receive results as text. (c) Use JPL swi-prolog.org/packages/jpl (haven't tried it myself, but reported to work).
Pipe query to prolog may not be the one for me. REST WS will help, but might become chatty. I did start evaluating jpl about 3 days ago and looks ok to me. Thank You.
Marking your answer. I tried summarizing the state of the question while marking it. Let me know if you have something to add to it.