栏目分类
热点资讯
你的位置:欧冠体育游戏手机登录注册 > 工商档案 > 记一次 .NET 某药品仓储打点体系 卡死阐发

工商档案

记一次 .NET 某药品仓储打点体系 卡死阐发

发布日期:2022-08-07 05:05    点击次数:80

一:背景 1. 讲故事

这个月初,有位同伙wx上找到我,说他的api过一段时光后,就会出现只要要求,没有照顾的环境,截图以下:

从同伙的形貌中看样子顺序是被什么货物卡住了,这类卡死的成就经管起来相对俭朴,接上去我就用 windbg 给巨匠阐发一下。

二:Windbg 阐发 1. Request 要求正在干吗?

既然同伙说 api 有 request 无 response,那怎么去验证同伙的话对纰谬呢?我们都晓得 .NET 用 HttpContext 来默示一个要求,意在言外就是可以或许去抓 HttpContext 下的时长属性,Netext 中有一个 !whttp 敕令可以或许协助我们。

0:000> !whttp HttpContext    Thread Time Out Running  Status Verb     Url 000000563bf803b0   42 00:01:50 00:01:24    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x-HN 000000563bf84660   -- 00:01:50 Finished    200 GET      http://xxx.com:30003/ 000000563c4a0470   51 00:01:50 00:00:12    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx2C 00000056bbf63590   30 00:01:50 00:02:41    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx-B2C 00000056bc82a038   -- 00:01:50 Finished    200 GET      http://localhost:30003/ 00000056bc84a3e8   44 00:01:50 00:00:51    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x 00000056bc8671c8   46 00:01:50 00:00:45    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx-B2C 000000573bf44698   35 00:01:50 00:02:39    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x 000000573bf483c0   33 00:01:50 00:02:41    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x-HN 000000573bf97e80   40 00:01:50 00:02:32    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=ZJB2C 000000573c583b08   -- 00:01:50 Finished    200 GET      http://localhost:30003/ 000000573c589ec8   -- 00:01:50 Finished    200 GET      http://xxx.com:30003/Wms/xxx/xxx/xxx 000000573c760e28   -- 00:01:50 Finished    200 POST     http://xxx.com:30003/Wms/xxx/xxx/xxx 000000573c95f990   48 00:01:50 00:00:31    200 POST     http://xxx.com:30003/Wms/Co妹妹on/xxx?xxx=xxx&xxx=x-HN 00000057bbf4f8e8   31 00:01:50 00:02:12    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x 00000057bc080340   50 00:01:50 00:00:19    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x 000000583c4aee80   43 00:01:50 00:01:11    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx2B 000000583c4d0c50   53 00:01:50 00:00:01    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx2B 00000058bbf8f1a0   34 00:01:50 00:02:22    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx2B 000000593bfe1758   41 00:01:50 00:01:22    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx2C 000000593c892160   -- 00:01:50 Finished    200 GET      http://xxx.com:30003/Wms/xxx/xxx/xxxJob 000000593ca813b0   45 00:01:50 00:00:30    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx-HN 000000593caa45d8   -- 00:01:50 Finished    200 GET      http://xxx.com:30003/ 00000059bc1ad808   32 00:01:50 00:01:45    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=xxx-B2C 00000059bc1c3d70   36 00:01:50 00:01:29    200 POST     http://xxx.com:30003/Wms/xxx/xxx?xxx=xxx&xxx=x  25 HttpContext object(s) found matching criteria 

从 Running 列可以或许看到大多要求都已经达到1分钟以上,这也验证了同伙所说的卡死成就,根据经历,可以或许取 Running 列中最大的 httpContext 所在的线程,也就是上面的 30 和 33 号线程, 看看它们都在干什么?

2. 探讨 Running 最长线程

接上去切到 30 和 33 号线程,看看它们的线程栈。

0:000> ~30s ntdll!NtWaitForSingleObject+0xa: 00007ffd`b81f024a c3              ret 0:030> !clrstack  OS Thread Id: 0x29d0 (30)         Child SP               IP Call Site 0000005acc3ac590 00007ffdb81f024a [PrestubMethodFrame: 0000005acc3ac590] xxx.xxx.RedisConnectionHelp.get_Instance() 0000005acc3ac850 00007ffd4dd78911 xxx.xxx.RedisCache..ctor(Int32, System.String) 0000005acc3ac8c0 00007ffd4dd78038 xxx.xxx.CacheByRedis.HashGet[[System.__Canon, mscorlib]](System.String, System.String, Int32) 0000005acc3ac968 00007ffdabef1f7c [StubHelperFrame: 0000005acc3ac968]  0000005acc3ac9c0 00007ffd4dd77f18 xxx.xxx.Cache.xxx.GetCacheNotAreaDataEntity[[System.__Canon, mscorlib]](System.String, System.String, System.String)  ...  0:030> ~33s ntdll!NtWaitForMultipleObjects+0xa: 00007ffd`b81f07ba c3              ret 0:033> !clrstack  OS Thread Id: 0x3ad4 (33)         Child SP               IP Call Site 0000005accabae90 00007ffdb81f07ba [GCFrame: 0000005accabae90]  0000005accabafb8 00007ffdb81f07ba [HelperMethodFrame_1OBJ: 0000005accabafb8] System.Threading.Monitor.ObjWait(Boolean, Int32,工商档案 System.Object) 0000005accabb0d0 00007ffdaac60d64 System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken) 0000005accabb160 00007ffdaac5b4bb System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken) 0000005accabb1d0 00007ffdab5a01d1 System.Threading.Tasks.Task.InternalWait(Int32, System.Threading.CancellationToken) 0000005accabb2a0 00007ffdab59cfa7 System.Threading.Tasks.Task`1[[System.__Canon, mscorlib]].GetResultxxx(Boolean) 0000005accabb2e0 00007ffd4d8d338f xxx.Config.xxx.Config`1[[System.__Canon, mscorlib]].GetConfig(xxx.Config.Model.ConfigListener, System.Func`2<xxx.Config.Request.GetConfigRequest,System.Threading.Tasks.Task`1<System.String>>) 0000005accabb340 00007ffd4d8d2f40 xxx.Config.xxx.Config`1[[System.__Canon, mscorlib]].get_Item(System.String, System.String) 0000005accabb3c0 00007ffd4dd78f7f xxx.Util.BaseConfig.get_GetRedisConn() 0000005accabb440 00007ffd4dd78e9c xxx.xxx.RedisConnectionHelp.GetConnectionString() 0000005accabb4a0 00007ffd4dd789cb xxx.xxx.RedisConnectionHelp..cctor() 0000005accabb940 00007ffdabef6953 [GCFrame: 0000005accabb940]  0000005accabc5b0 00007ffdabef6953 [PrestubMethodFrame: 0000005accabc5b0] xxx.xxx.RedisConnectionHelp.get_Instance() 0000005accabc870 00007ffd4dd78911 xxx.xxx.RedisCache..ctor(Int32, System.String) 0000005accabc8e0 00007ffd4dd78038 xxx.xxx.CacheByRedis.HashGet[[System.__Canon, mscorlib]](System.String, System.String, Int32) 0000005accabc988 00007ffdabef1f7c [StubHelperFrame: 0000005accabc988]  0000005accabc9e0 00007ffd4dd77f18 xxx.Core.Cache.xxx.GetCacheNotAreaDataEntity[[System.__Canon, mscorlib]](System.String, System.String, System.String) ... 

上面的信息不难缔造 30 号线程正卡在 RedisConnectionHelp.get_Instance() 处,33 号线已经进入了 RedisConnectionHelp.get_Instance() 编制中,最后在 GetConfig() 处等待 Result 的终局,按履向来说,30 号线程看样子正在锁等待, 33 号正在等待异步终局,接上去的冲破点就是探讨下 RedisConnectionHelp.Instance 处代码。

3. 寻找成就代码

接上去用反编译器材 ILSpy 找到成就代码。

public static class RedisConnectionHelp {  public static ConnectionMultiplexer Instance  {   get   {    if (_instance == null)    {     lock (Locker)     {      if (_instance == null