You can go with ServiceBuilder's JSON Web Services, they're not limited to database entities. You'd have to push down your implementation from the portlet layer to the business layer though (which might be a good way to separate UI (portlet) from business code.

Another option is to go the "Headless" route, which provides proper REST services, but again, you'd not implement them in your portlet, but would push out the implementation to a different layer.

Lastly, if you insist on keeping the logic within your portlet, you can use a portlet's resourceURL, which accepts and generates arbitrary data, types and formats, but you'll have to handle quite a bit of the serializing/deserializing yourself that you'd get for free if you would use one of the service techniques.