Counting blocks of text

Discussion in 'Perl Misc' started by laredotornado, Nov 2, 2010.

  1. Hi,

    I'm using Perl 5.8.9 for Mac 10.6.3. I am trying to parse log files
    (log4j v1.2.4) and count identical blocks of text in my file. So
    below is an example of the kinds of things I see in my log file
    (notice that some blocks of text span several lines). I would like to
    get identify the similar blocks of text and get a count of how many
    times each occurs in the file. Any ideas how I can do this?

    Thanks, - Dave


    =====================example contents of a file =====================
    ERROR com.myco.servlet.summer.SummerServlet - db gave null for
    galleryId: 4
    ERROR com.myco.servlet.common.SimpleLeadTrackerServlet - LEAD
    INSERTION FAILED: Laughlin null null
    com.ibatis.common.jdbc.exception.NestedSQLException:
    --- The error occurred in campaignQueries.xml.
    --- The error occurred while preparing the mapped statement for
    execution.
    --- Check the Campaign.getXmlBannerForSrcId.
    --- Cause: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    201)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForList(MappedStatement.java:
    139)
    at
    com.ibatis.sqlmap.engine.mapping.statement.CachingStatement.executeQueryForList(CachingStatement.java:
    97)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    567)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    541)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:
    118)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryForList(SqlMapClientImpl.java:
    94)
    at com.myco.database.DataInterface.doOperation(DataInterface.java:
    180)
    at com.myco.database.DataInterface.doFullOperation(DataInterface.java:
    158)
    at com.myco.database.DataInterface.getResults(DataInterface.java:132)
    at com.myco.database.DataInterface.getResults(DataInterface.java:357)
    at com.myco.database.DataInterface.getResults(DataInterface.java:349)
    at
    com.myco.servlet.common.SimpleLeadTrackerServlet.processRequest(SimpleLeadTrackerServlet.java:
    129)
    at
    com.myco.servlet.common.SimpleLeadTrackerServlet.doPost(SimpleLeadTrackerServlet.java:
    250)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:
    88)
    at
    org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
    76)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:344)
    at
    org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:
    110)
    at
    org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:
    84)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:
    98)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:
    95)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:
    79)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:
    55)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:
    36)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:
    188)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:
    106)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:
    80)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:
    150)
    at
    org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:
    237)
    at
    org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:
    167)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    Caused by: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.validateParameter(MappedStatement.java:
    245)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    167)
    ... 62 more
    ERROR com.myco.servlet.common.SimpleLeadTrackerServlet - LEAD
    INSERTION FAILED: Laughlin null null
    com.ibatis.common.jdbc.exception.NestedSQLException:
    --- The error occurred in campaignQueries.xml.
    --- The error occurred while preparing the mapped statement for
    execution.
    --- Check the Campaign.getXmlBannerForSrcId.
    --- Cause: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    201)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForList(MappedStatement.java:
    139)
    at
    com.ibatis.sqlmap.engine.mapping.statement.CachingStatement.executeQueryForList(CachingStatement.java:
    97)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    567)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    541)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:
    118)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryForList(SqlMapClientImpl.java:
    94)
    at com.myco.database.DataInterface.doOperation(DataInterface.java:
    180)
    at com.myco.database.DataInterface.doFullOperation(DataInterface.java:
    158)
    at com.myco.database.DataInterface.getResults(DataInterface.java:132)
    at com.myco.database.DataInterface.getResults(DataInterface.java:357)
    at com.myco.database.DataInterface.getResults(DataInterface.java:349)
    at
    com.myco.servlet.common.SimpleLeadTrackerServlet.processRequest(SimpleLeadTrackerServlet.java:
    129)
    at
    com.myco.servlet.common.SimpleLeadTrackerServlet.doPost(SimpleLeadTrackerServlet.java:
    250)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:
    88)
    at
    org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
    76)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:344)
    at
    org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:
    110)
    at
    org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:
    84)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:
    98)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:
    95)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:
    79)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:
    55)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:
    36)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:
    188)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:
    106)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:
    80)
    at org.springframework.security.web.FilterChainProxy
    $VirtualFilterChain.doFilter(FilterChainProxy.java:356)
    at
    org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:
    150)
    at
    org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:
    237)
    at
    org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:
    167)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    Caused by: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.validateParameter(MappedStatement.java:
    245)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    167)
    ... 62 more
    ERROR com.myco.servlet.search.GoogleMiniSearchProxy - search(String
    searchParams): Exception Thrown.
    java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:
    195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:525)
    at java.net.Socket.connect(Socket.java:475)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
    at sun.net.www.http.HttpClient.New(HttpClient.java:306)
    at sun.net.www.http.HttpClient.New(HttpClient.java:323)
    at
    sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:
    860)
    at
    sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:
    801)
    at
    sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:
    726)
    at
    sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:
    1049)
    at
    com.myco.servlet.search.GoogleMiniSearchProxy.search(GoogleMiniSearchProxy.java:
    88)
    at
    com.myco.servlet.search.GoogleMiniSearchProxy.search(GoogleMiniSearchProxy.java:
    70)
    at
    com.myco.servlet.search.GoogleProxyServlet.doPost(GoogleProxyServlet.java:
    40)
    at
    com.myco.servlet.search.GoogleProxyServlet.doGet(GoogleProxyServlet.java:
    33)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    ERROR com.myco.servlet.search.GoogleMiniSearchProxy -
    search(HttpServletRequest request): Exception Thrown.
    java.lang.Exception: Site search not available. Please try again
    later.
    at
    com.myco.servlet.search.GoogleMiniSearchProxy.search(GoogleMiniSearchProxy.java:
    100)
    at
    com.myco.servlet.search.GoogleMiniSearchProxy.search(GoogleMiniSearchProxy.java:
    70)
    at
    com.myco.servlet.search.GoogleProxyServlet.doPost(GoogleProxyServlet.java:
    40)
    at
    com.myco.servlet.search.GoogleProxyServlet.doGet(GoogleProxyServlet.java:
    33)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    ERROR com.myco.servlet.search.GoogleProxyServlet -
    post(request,response)
    java.lang.Exception: Site search not available. Please try again
    later.
    at
    com.myco.servlet.search.GoogleMiniSearchProxy.search(GoogleMiniSearchProxy.java:
    74)
    at
    com.myco.servlet.search.GoogleProxyServlet.doPost(GoogleProxyServlet.java:
    40)
    at
    com.myco.servlet.search.GoogleProxyServlet.doGet(GoogleProxyServlet.java:
    33)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    ERROR com.myco.servlet.common.LeadTrackerServlet2 - [processRequest]
    [sourceURL: not set]
    ERROR com.myco.servlet.common.LeadTrackerServlet2 - [processRequest]
    [LEAD INSERTION FAILED: ]
    com.ibatis.common.jdbc.exception.NestedSQLException:
    --- The error occurred in campaignQueries.xml.
    --- The error occurred while preparing the mapped statement for
    execution.
    --- Check the Campaign.getXmlBannerForSrcId.
    --- Cause: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    201)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForList(MappedStatement.java:
    139)
    at
    com.ibatis.sqlmap.engine.mapping.statement.CachingStatement.executeQueryForList(CachingStatement.java:
    97)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    567)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForList(SqlMapExecutorDelegate.java:
    541)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForList(SqlMapSessionImpl.java:
    118)
    at
    com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.queryForList(SqlMapClientImpl.java:
    94)
    at com.myco.database.DataInterface.doOperation(DataInterface.java:
    180)
    at com.myco.database.DataInterface.doFullOperation(DataInterface.java:
    158)
    at com.myco.database.DataInterface.getResults(DataInterface.java:132)
    at com.myco.database.DataInterface.getResults(DataInterface.java:357)
    at com.myco.database.DataInterface.getResults(DataInterface.java:349)
    at
    com.myco.servlet.common.LeadTrackerServlet2.processRequest(LeadTrackerServlet2.java:
    105)
    at
    com.myco.servlet.common.LeadTrackerServlet2.doPost(LeadTrackerServlet2.java:
    254)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    290)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at com.myco.filter.RecordPageFilter.doFilter(RecordPageFilter.java:
    35)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:
    235)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
    206)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:
    233)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:
    191)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:
    127)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:
    102)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
    109)
    at
    org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:
    298)
    at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:
    190)
    at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:
    291)
    at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:769)
    at
    org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:
    698)
    at org.apache.jk.common.ChannelSocket
    $SocketConnection.runIt(ChannelSocket.java:891)
    at org.apache.tomcat.util.threads.ThreadPool
    $ControlRunnable.run(ThreadPool.java:690)
    at java.lang.Thread.run(Thread.java:619)
    Caused by: java.sql.SQLException: Invalid parameter object type.
    Expected 'java.lang.Integer' but found 'java.util.HashMap'.
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.validateParameter(MappedStatement.java:
    245)
    at
    com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    167)
    ... 33 more
    ====================end example contents of a file ===================
    laredotornado, Nov 2, 2010
    #1
    1. Advertising

  2. laredotornado

    J. Gleixner Guest

    laredotornado wrote:
    > Hi,
    >
    > I'm using Perl 5.8.9 for Mac 10.6.3. I am trying to parse log files
    > (log4j v1.2.4) and count identical blocks of text in my file. So
    > below is an example of the kinds of things I see in my log file
    > (notice that some blocks of text span several lines). I would like to
    > get identify the similar blocks of text and get a count of how many
    > times each occurs in the file. Any ideas how I can do this?


    >
    > =====================example contents of a file =====================
    > ERROR com.myco.servlet.summer.SummerServlet - db gave null for
    > galleryId: 4
    > ERROR com.myco.servlet.common.SimpleLeadTrackerServlet - LEAD
    > INSERTION FAILED: Laughlin null null
    > com.ibatis.common.jdbc.exception.NestedSQLException:

    [... removed hundreds of useless errors... ]

    > at
    > com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    > 167)
    > ... 33 more

    You post hundreds of lines of errors, and don't include the last 33..
    Very thoughtful. Next time, there's no need to post that many lines,
    just post a few, showing different errors.

    You don't say what you have already tried, but before you start
    to write the code, first you need to define what you want to look
    for in the file, that would constitute an error. e.g. does every
    error start with 'ERROR '??, Does it start with a capital
    letter?, etc.

    Then you have to open the file and look for when that happens,
    and keep track of the error and how many times it happened.

    There are different ways to do it, but in general you'd:

    open the file.
    while you read the file, possibly line by line....
    if the line contains the indicator of a new error.
    if there was already a previous error that was found, then
    store that previous, error.. maybe in a hash, and
    increment the count.. e.g. $hash{ $error }++;
    store the line as an error. $error = $_
    else append the line to the previous error e.g. $error .= $_
    close the file.
    Be sure to capture the last error.. $hash{ $error }++;

    Iterate through the keys in %hash, which are really the text of the
    error, to find the error and how many times it occurred.

    perldoc perlopentut
    perldoc perlretut
    perldoc -q 'How do I process an entire hash'
    J. Gleixner, Nov 2, 2010
    #2
    1. Advertising

  3. laredotornado

    Guest

    On Tue, 02 Nov 2010 10:32:40 -0500, "J. Gleixner" <> wrote:

    >laredotornado wrote:
    >> Hi,
    >>
    >> I'm using Perl 5.8.9 for Mac 10.6.3. I am trying to parse log files
    >> (log4j v1.2.4) and count identical blocks of text in my file. So
    >> below is an example of the kinds of things I see in my log file
    >> (notice that some blocks of text span several lines). I would like to
    >> get identify the similar blocks of text and get a count of how many
    >> times each occurs in the file. Any ideas how I can do this?

    >
    >>
    >> =====================example contents of a file =====================
    >> ERROR com.myco.servlet.summer.SummerServlet - db gave null for
    >> galleryId: 4
    >> ERROR com.myco.servlet.common.SimpleLeadTrackerServlet - LEAD
    >> INSERTION FAILED: Laughlin null null
    >> com.ibatis.common.jdbc.exception.NestedSQLException:

    >[... removed hundreds of useless errors... ]
    >
    >> at
    >> com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:
    >> 167)
    >> ... 33 more

    >You post hundreds of lines of errors, and don't include the last 33..
    >Very thoughtful. Next time, there's no need to post that many lines,
    >just post a few, showing different errors.
    >
    >You don't say what you have already tried, but before you start
    >to write the code, first you need to define what you want to look
    >for in the file, that would constitute an error. e.g. does every
    >error start with 'ERROR '??, Does it start with a capital
    >letter?, etc.
    >
    >Then you have to open the file and look for when that happens,
    >and keep track of the error and how many times it happened.
    >
    >There are different ways to do it, but in general you'd:
    >
    >open the file.
    >while you read the file, possibly line by line....
    > if the line contains the indicator of a new error.
    > if there was already a previous error that was found, then
    > store that previous, error.. maybe in a hash, and
    > increment the count.. e.g. $hash{ $error }++;
    > store the line as an error. $error = $_
    > else append the line to the previous error e.g. $error .= $_
    >close the file.
    >Be sure to capture the last error.. $hash{ $error }++;
    >
    >Iterate through the keys in %hash, which are really the text of the
    >error, to find the error and how many times it occurred.
    >
    >perldoc perlopentut
    >perldoc perlretut
    >perldoc -q 'How do I process an entire hash'
    >


    You may want to add some bigger indenting to separate
    the apparent inner if/else from the apparent outer
    if/else. There is only one else.

    > if the line contains the indicator of a new error.
    >
    > if there was already a previous error that was found, then
    > store that previous, error.. maybe in a hash, and
    > increment the count.. e.g. $hash{ $error }++;
    > store the line as an error. $error = $_
    >
    > else
    > append the line to the previous error e.g. $error .= $_


    Still, ERROR
    ...
    ...
    to ERROR

    will consume the entire log file as chopped up hash keys.

    -sln
    , Nov 3, 2010
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Brent

    Counting lines of text.

    Brent, Feb 16, 2005, in forum: ASP .Net
    Replies:
    3
    Views:
    387
    =?Utf-8?B?TmlrYW5kZXIgJiBNYXJncmlldCBCcnVnZ2VtYW4=
    Feb 18, 2005
  2. Arjen
    Replies:
    3
    Views:
    427
    Scott Allen
    Feb 27, 2005
  3. matt
    Replies:
    1
    Views:
    240
    George Ogata
    Aug 6, 2004
  4. Steven Taylor
    Replies:
    9
    Views:
    236
    Brian Candler
    Apr 27, 2009
  5. edwardfredriks

    counting up instead of counting down

    edwardfredriks, Sep 6, 2005, in forum: Javascript
    Replies:
    6
    Views:
    186
    Dr John Stockton
    Sep 7, 2005
Loading...

Share This Page