最近作的一个项目由于安全审计须要,须要作安全改造。其中天然就包括XSS和CSRF漏洞安全整改。关于这两个网络安全漏洞的详细说明,能够参照我本篇博客最后的参考连接。固然,我这里并非想写一篇安全方面的专题。我要讲的是在作了XSS漏洞修复以后引起的一系列事件。java
本地测试的时候随便点了些页面,而后debug跟了下代码未发现任何问题。上线以后用户反馈有的页面打不开,本身去线上体验发现大部分页面正常,可是存在部分客户反馈的页面打开直接超时报错。ios
XSS这个漏洞修复开始不是通过我处理的,上线以后因为编码规则太严格(后面我会讲咱们使用的解决方案),致使前台传入的JSON字符串中的引号全被转码,形成后台解析报错。web
而我成为了那个救(bei)火(guo)英(xia)雄,须要立马解决这个问题补丁升级。我看了下之前实现的代码,有考虑到经过一个XML白名单文件,配置忽略XSS编码的请求URI。最直接的办法,是直接把我这个请求的URI加入XML白名单配置,而后补丁替换白名单文件重启服务。spring
可是当时我在修改的时候,考虑到可能不止这一个须要过滤的白名单,若是纯粹启动时加载的XML白名单列表。到时候还有别的URI须要忽略,那我岂不是还要再发增量补丁......数据库
因而当时我修改的时候顺便增长了一个能力,白名单能够直接在界面配置,而且每次获取白名单列表的时候动态从数据库获取(考虑到实时请求较大,我调用的系统已有接口提供的支持缓存的查询方法)。正由于有这个“后门”我直接在线上配置了这个参数,先暂且把线上问题解决。express
解决问题第一步,天然是分析线上日志。发现线上日志的确对请求的URI中的参数作了XSS编码处理。编程
那问题就回到咱们XSS漏洞修复的实现方式:AntiSamy(见参考连接)。其中咱们的XssRequestWrapper
源码以下:json
public class XssRequestWrapper extends HttpServletRequestWrapper { private static Logger log = LoggerFactory.getLogger(XssRequestWrapper.class); private static Policy policy = null; static { String path = XssRequestWrapper.class.getClassLoader().getResource("security/antisamy-tinymce.xml").getFile(); log.info("policy_filepath:" + path); if (path.startsWith("file")) { //以file:开头 path = path.substring(6); } try { policy = Policy.getInstance(path); } catch (PolicyException e) { e.printStackTrace(); } } public XssRequestWrapper(HttpServletRequest request) { super(request); } // 队请求参数进行安全转码 public String getParameter(String paramString) { String str = super.getParameter(paramString); if (StringUtils.isBlank(str)) { return null; } return xssClean(str, paramString); } // 队请求头进行安全转码 public String getHeader(String paramString) { String str = super.getHeader(paramString); if (StringUtils.isBlank(str)) return null; return xssClean(str, paramString); } @SuppressWarnings({"rawtypes","unchecked"}) public Map<String, String[]> getParameterMap() { Map<String, String[]> request_map = super.getParameterMap(); Iterator iterator = request_map.entrySet().iterator(); log.debug("getParameterMap size:{}", request_map.size()); while (iterator.hasNext()) { Map.Entry me = (Map.Entry) iterator.next(); String paramsKey = (String) me.getKey(); String[] values = (String[]) me.getValue(); for (int i = 0; i < values.length; i++) { values[i] = xssClean(values[i], paramsKey); } } return request_map; } public String[] getParameterValues(String paramString) { String[] arrayOfString1 = super.getParameterValues(paramString); if (arrayOfString1 == null) return null; int i = arrayOfString1.length; String[] arrayOfString2 = new String[i]; for (int j = 0; j < i; j++) arrayOfString2[j] = xssClean(arrayOfString1[j], paramString); return arrayOfString2; } public final static String KEY_FILTER_STR = "'"; public final static String SUFFIX = "value"; //须要过滤的地方 private String xssClean(String value, String paramsKey) { String keyFilterStr = KEY_FILTER_STR; String param = paramsKey.toLowerCase(); if (param.endsWith(SUFFIX)) { // 若是参数名 name="xxxvalue" if (value.contains(keyFilterStr)) { value = value.replace(keyFilterStr, "‘"); } } AntiSamy antiSamy = new AntiSamy(); try { final CleanResults cr = antiSamy.scan(value, policy); // 安全的HTML输出 return cr.getCleanHTML(); } catch (ScanException e) { log.error("antiSamy scan error", e); } catch (PolicyException e) { log.error("antiSamy policy error", e); } return value; } }
能够看到了咱们项目组最终采用的策略配置文件是:antisamy-tinymce.xml
,这种策略只容许传送纯文本到后台(这样作真的好吗?我的以为这个规则太过严格),而且对请求头和请求参数都作了XSS转码。请注意这里,咱们相对于参考连接中源码不一样的处理方式在于:咱们对请求头也进行了编码处理。跨域
那么看来问题就在于编码致使的效率低下,因而我在getHeader
和getParameter
方法中都打了断点。在揭晓结果以前,我说说我当时的猜想:由于当时用户反馈的有问题的页面是有不少查询条件的,我开始的猜想是应该是传入后台的参数过多致使编码影响效率。然而,现实老是无情地打脸,无论你天真不天真。缓存
Debug发现getParameter
调用的此时算正常,而getHeader
处的断点没完没了的进来(最终结果证实,进来了几千次)......
仍是那句话,没有什么是源码解决不了的,若是有,那么请Debug源码。😄
咱们项目是传统的SpringMVC项目,那么固然咱们要从org.springframework.web.servlet.DispatcherServlet
入手了。DispatcherServlet
其实也是一个Servlet,他的继承关系以下:
DispatcherServlet extends FrameworkServlet FrameworkServlet extends HttpServletBean implements ApplicationContextAware HttpServletBean extends HttpServlet implements EnvironmentCapable, EnvironmentAware HttpServlet extends GenericServlet GenericServlet implements Servlet, ServletConfig, java.io.Serializable
能够看到,实际上SpringMVC的DispatcherServlet
最终也是经过doGet
和doPost
来对请求进行转发,而最终其实都到了DispatcherServlet
的doService
.该方法的源码以下:
/** * Exposes the DispatcherServlet-specific request attributes and delegates to {@link #doDispatch} * for the actual dispatching. */ @Override protected void doService(HttpServletRequest request, HttpServletResponse response) throws Exception { if (logger.isDebugEnabled()) { String resumed = WebAsyncUtils.getAsyncManager(request).hasConcurrentResult() ? " resumed" : ""; logger.debug("DispatcherServlet with name '" + getServletName() + "'" + resumed + " processing " + request.getMethod() + " request for [" + getRequestUri(request) + "]"); } // Keep a snapshot of the request attributes in case of an include, // to be able to restore the original attributes after the include. Map<String, Object> attributesSnapshot = null; if (WebUtils.isIncludeRequest(request)) { attributesSnapshot = new HashMap<String, Object>(); Enumeration<?> attrNames = request.getAttributeNames(); while (attrNames.hasMoreElements()) { String attrName = (String) attrNames.nextElement(); if (this.cleanupAfterInclude || attrName.startsWith("org.springframework.web.servlet")) { attributesSnapshot.put(attrName, request.getAttribute(attrName)); } } } // Make framework objects available to handlers and view objects. request.setAttribute(WEB_APPLICATION_CONTEXT_ATTRIBUTE, getWebApplicationContext()); request.setAttribute(LOCALE_RESOLVER_ATTRIBUTE, this.localeResolver); request.setAttribute(THEME_RESOLVER_ATTRIBUTE, this.themeResolver); request.setAttribute(THEME_SOURCE_ATTRIBUTE, getThemeSource()); FlashMap inputFlashMap = this.flashMapManager.retrieveAndUpdate(request, response); if (inputFlashMap != null) { request.setAttribute(INPUT_FLASH_MAP_ATTRIBUTE, Collections.unmodifiableMap(inputFlashMap)); } request.setAttribute(OUTPUT_FLASH_MAP_ATTRIBUTE, new FlashMap()); request.setAttribute(FLASH_MAP_MANAGER_ATTRIBUTE, this.flashMapManager); try { // 重点在这,都会进入doDispatch方法 doDispatch(request, response); } finally { if (!WebAsyncUtils.getAsyncManager(request).isConcurrentHandlingStarted()) { // Restore the original attribute snapshot, in case of an include. if (attributesSnapshot != null) { restoreAttributesAfterInclude(request, attributesSnapshot); } } } }
能够看到doService
方法,最终仍是会进入到doDispatch
中,该方法的源码以下:
/** * Process the actual dispatching to the handler. * <p>The handler will be obtained by applying the servlet's HandlerMappings in order. * The HandlerAdapter will be obtained by querying the servlet's installed HandlerAdapters * to find the first that supports the handler class. * <p>All HTTP methods are handled by this method. It's up to HandlerAdapters or handlers * themselves to decide which methods are acceptable. * @param request current HTTP request * @param response current HTTP response * @throws Exception in case of any kind of processing failure */ protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception { HttpServletRequest processedRequest = request; HandlerExecutionChain mappedHandler = null; boolean multipartRequestParsed = false; WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request); try { ModelAndView mv = null; Exception dispatchException = null; try { processedRequest = checkMultipart(request); multipartRequestParsed = (processedRequest != request); // Determine handler for the current request. // 根据请求获取处理的Handler mappedHandler = getHandler(processedRequest); if (mappedHandler == null || mappedHandler.getHandler() == null) { noHandlerFound(processedRequest, response); return; } // Determine handler adapter for the current request. HandlerAdapter ha = getHandlerAdapter(mappedHandler.getHandler()); // Process last-modified header, if supported by the handler. String method = request.getMethod(); boolean isGet = "GET".equals(method); if (isGet || "HEAD".equals(method)) { long lastModified = ha.getLastModified(request, mappedHandler.getHandler()); if (logger.isDebugEnabled()) { logger.debug("Last-Modified value for [" + getRequestUri(request) + "] is: " + lastModified); } if (new ServletWebRequest(request, response).checkNotModified(lastModified) && isGet) { return; } } if (!mappedHandler.applyPreHandle(processedRequest, response)) { return; } // Actually invoke the handler. mv = ha.handle(processedRequest, response, mappedHandler.getHandler()); if (asyncManager.isConcurrentHandlingStarted()) { return; } applyDefaultViewName(processedRequest, mv); mappedHandler.applyPostHandle(processedRequest, response, mv); } catch (Exception ex) { dispatchException = ex; } catch (Throwable err) { // As of 4.3, we're processing Errors thrown from handler methods as well, // making them available for @ExceptionHandler methods and other scenarios. dispatchException = new NestedServletException("Handler dispatch failed", err); } processDispatchResult(processedRequest, response, mappedHandler, mv, dispatchException); } catch (Exception ex) { triggerAfterCompletion(processedRequest, response, mappedHandler, ex); } catch (Throwable err) { triggerAfterCompletion(processedRequest, response, mappedHandler, new NestedServletException("Handler processing failed", err)); } finally { if (asyncManager.isConcurrentHandlingStarted()) { // Instead of postHandle and afterCompletion if (mappedHandler != null) { mappedHandler.applyAfterConcurrentHandlingStarted(processedRequest, response); } } else { // Clean up any resources used by a multipart request. if (multipartRequestParsed) { cleanupMultipart(processedRequest); } } } }
这段处理的重点也在我中文注释的地方,咱们继续跟进getHandler
方法:
protected HandlerExecutionChain getHandler(HttpServletRequest request) throws Exception { // 遍历HandlerMapping,找到请求对应的Handler具体信息 for (HandlerMapping hm : this.handlerMappings) { if (logger.isTraceEnabled()) { logger.trace( "Testing handler map [" + hm + "] in DispatcherServlet with name '" + getServletName() + "'"); } HandlerExecutionChain handler = hm.getHandler(request); if (handler != null) { return handler; } } return null; } // 最终会到AbstractHandlerMethodMapping#getHandlerInternal protected HandlerMethod getHandlerInternal(HttpServletRequest request) throws Exception { // 分析请求URI (去掉Context) String lookupPath = getUrlPathHelper().getLookupPathForRequest(request); if (logger.isDebugEnabled()) { logger.debug("Looking up handler method for path " + lookupPath); } this.mappingRegistry.acquireReadLock(); try { // 重头戏: 根据请求URI去找对应的处理器 (具体到类和方法) HandlerMethod handlerMethod = lookupHandlerMethod(lookupPath, request); if (logger.isDebugEnabled()) { if (handlerMethod != null) { logger.debug("Returning handler method [" + handlerMethod + "]"); } else { logger.debug("Did not find handler method for [" + lookupPath + "]"); } } return (handlerMethod != null ? handlerMethod.createWithResolvedBean() : null); } finally { this.mappingRegistry.releaseReadLock(); } }
注意其中的中文注释,咱们经过解析请求中的URI,而后根据请求URI去查找对应的处理器,也就是进行适配的关键一步:
/** * Look up the best-matching handler method for the current request. * If multiple matches are found, the best match is selected. * @param lookupPath mapping lookup path within the current servlet mapping * @param request the current request * @return the best-matching handler method, or {@code null} if no match * @see #handleMatch(Object, String, HttpServletRequest) * @see #handleNoMatch(Set, String, HttpServletRequest) */ protected HandlerMethod lookupHandlerMethod(String lookupPath, HttpServletRequest request) throws Exception { List<Match> matches = new ArrayList<Match>(); // 直接根据请求的URI去找对应的处理类(此时前台请求URI必须与后台注解配置的RequestMapping彻底一致) List<T> directPathMatches = this.mappingRegistry.getMappingsByUrl(lookupPath); if (directPathMatches != null) { addMatchingMappings(directPathMatches, matches, request); } if (matches.isEmpty()) { // 注意这里: SpringMVC作了个无奈的容错处理,若是没有彻底匹配的话,就遍历全部请求URI找到大体匹配的 --- 这也是本次问题出现的缘由 // No choice but to go through all mappings... addMatchingMappings(this.mappingRegistry.getMappings().keySet(), matches, request); } if (!matches.isEmpty()) { Comparator<Match> comparator = new MatchComparator(getMappingComparator(request)); Collections.sort(matches, comparator); if (logger.isTraceEnabled()) { logger.trace("Found " + matches.size() + " matching mapping(s) for [" + lookupPath + "] : " + matches); } Match bestMatch = matches.get(0); if (matches.size() > 1) { if (CorsUtils.isPreFlightRequest(request)) { return PREFLIGHT_AMBIGUOUS_MATCH; } Match secondBestMatch = matches.get(1); if (comparator.compare(bestMatch, secondBestMatch) == 0) { Method m1 = bestMatch.handlerMethod.getMethod(); Method m2 = secondBestMatch.handlerMethod.getMethod(); throw new IllegalStateException("Ambiguous handler methods mapped for HTTP path '" + request.getRequestURL() + "': {" + m1 + ", " + m2 + "}"); } } handleMatch(bestMatch.mapping, lookupPath, request); return bestMatch.handlerMethod; } else { return handleNoMatch(this.mappingRegistry.getMappings().keySet(), lookupPath, request); } }
其实到这里,咱们已经找到真相了,可是为何循环遍历请求URI会致使getHeader
方法超频调用呢?咱们继续跟进:
private void addMatchingMappings(Collection<T> mappings, List<Match> matches, HttpServletRequest request) { for (T mapping : mappings) { T match = getMatchingMapping(mapping, request); if (match != null) { matches.add(new Match(match, this.mappingRegistry.getMappings().get(mapping))); } } } // RequestMappingInfoHandlerMapping /** * Check if the given RequestMappingInfo matches the current request and * return a (potentially new) instance with conditions that match the * current request -- for example with a subset of URL patterns. * @return an info in case of a match; or {@code null} otherwise. */ @Override protected RequestMappingInfo getMatchingMapping(RequestMappingInfo info, HttpServletRequest request) { return info.getMatchingCondition(request); } // RequestMappingInfo /** * Checks if all conditions in this request mapping info match the provided request and returns * a potentially new request mapping info with conditions tailored to the current request. * <p>For example the returned instance may contain the subset of URL patterns that match to * the current request, sorted with best matching patterns on top. * @return a new instance in case all conditions match; or {@code null} otherwise */ @Override public RequestMappingInfo getMatchingCondition(HttpServletRequest request) { RequestMethodsRequestCondition methods = this.methodsCondition.getMatchingCondition(request); ParamsRequestCondition params = this.paramsCondition.getMatchingCondition(request); // 问题就在于headersCondition的 getMatchingCondition 方法的调用 HeadersRequestCondition headers = this.headersCondition.getMatchingCondition(request); ConsumesRequestCondition consumes = this.consumesCondition.getMatchingCondition(request); ProducesRequestCondition produces = this.producesCondition.getMatchingCondition(request); if (methods == null || params == null || headers == null || consumes == null || produces == null) { return null; } PatternsRequestCondition patterns = this.patternsCondition.getMatchingCondition(request); if (patterns == null) { return null; } RequestConditionHolder custom = this.customConditionHolder.getMatchingCondition(request); if (custom == null) { return null; } return new RequestMappingInfo(this.name, patterns, methods, params, headers, consumes, produces, custom.getCondition()); } // HeadersRequestCondition public HeadersRequestCondition getMatchingCondition(HttpServletRequest request) { // 最终调用的是CorsUtils#isCorsRequest if (CorsUtils.isPreFlightRequest(request)) { return PRE_FLIGHT_MATCH; } for (HeaderExpression expression : expressions) { if (!expression.match(request)) { return null; } } return this; } // CorsUtils /** * Returns {@code true} if the request is a valid CORS one. */ public static boolean isCorsRequest(HttpServletRequest request) { // XSS的编码是经过Filter对请求中的Header进行编码的,因此每次遍历URI都会调用一次请求头编码 return (request.getHeader(HttpHeaders.ORIGIN) != null); } /** * Returns {@code true} if the request is a valid CORS pre-flight one. */ public static boolean isPreFlightRequest(HttpServletRequest request) { return (isCorsRequest(request) && HttpMethod.OPTIONS.matches(request.getMethod()) && request.getHeader(HttpHeaders.ACCESS_CONTROL_REQUEST_METHOD) != null); }
好了,真相终于浮出水面了。就是由于若是前台请求URI没有彻底匹配后台配置的话会致使每次跨域请求校验都会对请求头中参数进行编码。而项目中请求的URI有几千个,假设每一个请求头平均有3个参数。那么,一次请求编码可能上万次...... 你说超时不超时?
最终和领导讨论确认去掉对header的XSS编码处理。咱们业务上不存在将Header的参数入库的状况。
善于思考的小伙伴必定会问了: 为何会有请求URI不匹配呢?若是不匹配之前为何能正常请求到呢?
哈哈,咱们继续看下文:
经过最后的getMatchingCondition
方法,咱们能够看到要想最终能找到一个匹配的请求的URI。上面几个condition必须至少要知足一个。经过debug我发现,最终在咱们项目中匹配的是patternsCondition
。那么这condition的具体实现是咋样的呢?直接见源码:
/** * Checks if any of the patterns match the given request and returns an instance * that is guaranteed to contain matching patterns, sorted via * {@link PathMatcher#getPatternComparator(String)}. * <p>A matching pattern is obtained by making checks in the following order: * <ul> * <li>Direct match * <li>Pattern match with ".*" appended if the pattern doesn't already contain a "." * <li>Pattern match * <li>Pattern match with "/" appended if the pattern doesn't already end in "/" * </ul> * @param request the current request * @return the same instance if the condition contains no patterns; * or a new condition with sorted matching patterns; * or {@code null} if no patterns match. */ public PatternsRequestCondition getMatchingCondition(HttpServletRequest request) { if (this.patterns.isEmpty()) { return this; } String lookupPath = this.pathHelper.getLookupPathForRequest(request); List<String> matches = getMatchingPatterns(lookupPath); return matches.isEmpty() ? null : new PatternsRequestCondition(matches, this.pathHelper, this.pathMatcher, this.useSuffixPatternMatch, this.useTrailingSlashMatch, this.fileExtensions); } /** * Find the patterns matching the given lookup path. Invoking this method should * yield results equivalent to those of calling * {@link #getMatchingCondition(javax.servlet.http.HttpServletRequest)}. * This method is provided as an alternative to be used if no request is available * (e.g. introspection, tooling, etc). * @param lookupPath the lookup path to match to existing patterns * @return a collection of matching patterns sorted with the closest match at the top */ public List<String> getMatchingPatterns(String lookupPath) { List<String> matches = new ArrayList<String>(); for (String pattern : this.patterns) { String match = getMatchingPattern(pattern, lookupPath); if (match != null) { matches.add(match); } } Collections.sort(matches, this.pathMatcher.getPatternComparator(lookupPath)); return matches; } private String getMatchingPattern(String pattern, String lookupPath) { if (pattern.equals(lookupPath)) { return pattern; } // 若是使用后缀匹配模式 且后台配置的URI没有后缀(不包含.),且前台请求的URI中包含. 则在后台配置的URI原来的匹配模式上加上 .* 再与前台请求URI进行匹配进行匹配 if (this.useSuffixPatternMatch) { if (!this.fileExtensions.isEmpty() && lookupPath.indexOf('.') != -1) { for (String extension : this.fileExtensions) { if (this.pathMatcher.match(pattern + extension, lookupPath)) { return pattern + extension; } } } else { boolean hasSuffix = pattern.indexOf('.') != -1; if (!hasSuffix && this.pathMatcher.match(pattern + ".*", lookupPath)) { return pattern + ".*"; } } } if (this.pathMatcher.match(pattern, lookupPath)) { return pattern; } if (this.useTrailingSlashMatch) { if (!pattern.endsWith("/") && this.pathMatcher.match(pattern + "/", lookupPath)) { return pattern +"/"; } } return null; }
注意中文注释的部分。咱们项目中这种不匹配的状况是后台只配置了/aaa/aaaa
而前台配置的是/aaa/aaaa.json
,天然符合前面的模式匹配。天然,也就不会匹配不到后台请求。
每一次采坑,都可以让人前进。经过此次XSS事件,我这边仍是有很多收获的:
- 多想一想后门: 实现功能的时候要多考虑可能出现的意外状况,实现功能很简单,解决功能可能致使的问题倒是一个值的深刻思考的方向
- 源码debug的重要性: 当有必定的编程经验以后,不少问题须要耐心debug才能解决。经过这个debug的过程,不只可以了解底层的实现流程,也熟悉了大神写的代码。 何乐而不为呢?
- SpringMVC前台请求最好与后台配置彻底匹配: 经过这篇文章相信你们也看到了,不匹配的后果。SpringMVC源码中的省略号已经充分展现了他对你这种行为的无语 😢
此次的博客就到这里,一是篇幅太长了。怕吃太多消化不良,二来留点悬念,咱们下回分解(虽然不知道下回是啥时候了,哈哈哈 我尽快.....)
AntiSamy: https://blog.csdn.net/qq_35946990/article/details/74982760
XSS与CSRF: https://www.jianshu.com/p/64a413ada155