How to prevent user from redirecting to Error page in MVC 4

I have a created a custom exception handler to handle all unhandled exceptions in MVC application and logging error details in database.

 

The problem is -it logs the exception but also redirects to Error page.

 

I want user to stay on the same view whenever error comes and some generic message should be displayed to the user like 'Error while saving data'. 

 

 I have removed defaultRedirect="Error" from below setting but still it redirects 

 

How to achieve this functionality.