So you need to figure out whether the issue is in the data retrieval from the source database or if it is in the rendering time. You can find this out by looking in the ExecutionLog3 view in the ReportServer database.
If most of the time is spent on Data Retrieval then you need to look at tuning your queries. If most of the time is spent on the rendering then you either need to look at simplifying the layout and formatting or reducing the number of rows. That is a very large number of rows and it sounds more like it is a data extract, not a report that someone would read through line by line. So you may be better off using a different approach if what you are doing is producing data extracts.
