Edit: Based on @Aaronaught great answer I'm adding few points targeting performance with EF. Those new points are prefixed by Edit.

The biggest improvement in performance in high traffic websites is achieved by caching (= first of all avoiding any web server processing or database querying) followed by asynchronous processing to avoid thread blocking while database queries are performed.

There is no bullet proof answer to your question because it always depends on requirements for application and on complexity of queries. The truth is that developer productivity with EF hides complexity behind which in many cases leads to incorrect usage of EF and terrible performance. The idea that you can expose high level abstracted interface for data access and it will smoothly works in all cases doesn't work. Even with ORM you must know what is happening behind the abstraction and how to correctly use it.

If you don't have previous experience with EF you will meet a lot of challenges when dealing with performance. You can make much more mistakes when working with EF comparing to ADO.NET. Also there is a lot of additional processing done in EF, so EF will always be significantly slower than native ADO.NET - that is something you can measure by simple proof of concept application.

If you want to get best performance from EF you will most probably have to:

• Very carefully revise your data access with SQL profiler and review your LINQ queries if they correctly use Linq-to-entities instead of Linq-to-objects
• Very carefully use advanced EF optimization features like MergeOption.NoTracking
• Use ESQL in some cases
• Pre-compile queries which are executed often
• Think about taking advantage of EF Caching wrapper to get "second level cache" like feature for some queries
• Use SQL views or custom mapped SQL queries (requires manual maintaining of EDMX file) in some scenarios for often used projections or aggregations which needs performance improvements
• Use native SQL and stored procedures for some queries which don't provide sufficient performance when defined in Linq or ESQL
• Edit: Carefully use queries - every query makes separate roundtrip to the database. EFv4 has no query batching because it is not able to use multiple result sets per executed database command. EFv4.5 will support multiple result sets for mapped stored procedures.
• Edit: Carefully work with data modifications. Again EF completely lacks command batching. So in ADO.NET you can use single SqlCommand containing multiple insert, updates or deletes but with EF every such command will be executed in separate roundtrip to database.
• Edit: Carefully work with identity map / identity cache. EF has special method (GetByKey in ObjectContext API or Find in DbContext API) to query the cache first. If you use Linq-to-entities or ESQL it will create roundtrip to the database and after that it will return existing instance from the cache.
• Edit: Carefully use eager loading. It is not always win-win solution because it produces one huge dataset. As you can see it is a lot of additional complexity and that is the whole point. ORM makes mapping and materialization simpler but when dealing with performance it will make it much more complex and you will have to make trade-offs.

I'm not sure if SO is still using L2S. They developed new open source ORM called Dapper and I think the main point behind this development was increasing performance.

Edit: Based on @Aaronaught great answer I'm adding few points targeting performance with EF. Those new points are prefixed by Edit.

The biggest improvement in performance in high traffic websites is achieved by caching (= first of all avoiding any web server processing or database querying) followed by asynchronous processing to avoid thread blocking while database queries are performed.

There is no bullet proof answer to your question because it always depends on requirements for application and on complexity of queries. The truth is that developer productivity with EF hides complexity behind which in many cases leads to incorrect usage of EF and terrible performance. The idea that you can expose high level abstracted interface for data access and it will smoothly works in all cases doesn't work. Even with ORM you must know what is happening behind the abstraction and how to correctly use it.

If you don't have previous experience with EF you will meet a lot of challenges when dealing with performance. You can make much more mistakes when working with EF comparing to ADO.NET. Also there is a lot of additional processing done in EF, so EF will always be significantly slower than native ADO.NET - that is something you can measure by simple proof of concept application.

If you want to get best performance from EF you will most probably have to:

• Very carefully revise your data access with SQL profiler and review your LINQ queries if they correctly use Linq-to-entities instead of Linq-to-objects
• Very carefully use advanced EF optimization features like MergeOption.NoTracking
• Use ESQL in some cases
• Pre-compile queries which are executed often
• Think about taking advantage of EF Caching wrapper to get "second level cache" like feature for some queries
• Use SQL views or custom mapped SQL queries (requires manual maintaining of EDMX file) in some scenarios for often used projections or aggregations which needs performance improvements
• Use native SQL and stored procedures for some queries which don't provide sufficient performance when defined in Linq or ESQL
• Edit: Carefully use queries - every query makes separate roundtrip to the database. EFv4 has no query batching because it is not able to use multiple result sets per executed database command. EFv4.5 will support multiple result sets for mapped stored procedures.
• Edit: Carefully work with data modifications. Again EF completely lacks command batching. So in ADO.NET you can use single SqlCommand containing multiple insert, updates or deletes but with EF every such command will be executed in separate roundtrip to database.
• Edit: Carefully work with identity map / identity cache. EF has special method (GetByKey in ObjectContext API or Find in DbContext API) to query the cache first. If you use Linq-to-entities or ESQL it will create roundtrip to the database and after that it will return existing instance from the cache.
• Edit: Carefully use eager loading. It is not always win-win solution because it produces one huge dataset. As you can see it is a lot of additional complexity and that is the whole point. ORM makes mapping and materialization simpler but when dealing with performance it will make it much more complex and you will have to make trade-offs.

I'm not sure if SO is still using L2S. They developed new open source ORM called Dapper and I think the main point behind this development was increasing performance.