mirror of
https://github.com/wabbajack-tools/wabbajack.git
synced 2024-08-30 18:42:17 +00:00
Merge remote-tracking branch 'erri/master' into compiler-multi-source
This commit is contained in:
Justin Swanson
commit
d1806e0ac3
@ -1,6 +1,6 @@
|
||||
# Contributing to Wabbajack
|
||||
|
||||
The following is a set of guidelines for contributing to the `wabbajack-tools/wabbajack` repo on GitHub. These are guidelines but not rules so be free to propose changes.
|
||||
The following is a set of guidelines for contributing to the `wabbajack-tools/wabbajack` repo on GitHub. These are guidelines but not rules, so be free to propose changes.
|
||||
|
||||
## How Can I Contribute?
|
||||
|
||||
@ -8,42 +8,42 @@ You don't have to be a programmer to contribute to this project.
|
||||
|
||||
### Reporting Bugs
|
||||
|
||||
When you encounter problems with the application, go to our [discord](https://discord.gg/zgbrkmA) server first and ask for help there. Before creating a new Issue, take a look at the others to avoid getting the [Duplicate](https://github.com/wabbajack-tools/wabbajack/labels/duplicate) label.
|
||||
When you encounter problems with the application, go to our [Discord](https://discord.gg/zgbrkmA) server first and ask for help there. Before creating a new Issue, take a look at the others to avoid getting the [Duplicate](https://github.com/wabbajack-tools/wabbajack/labels/duplicate) label.
|
||||
|
||||
Creating a bug report is as easy as navigating to the [Issues](https://github.com/wabbajack-tools/wabbajack/issues) page and clicking the [New Issue](https://github.com/wabbajack-tools/wabbajack/issues/new/choose) button.
|
||||
|
||||
#### Submitting A Good Bug Report
|
||||
|
||||
* Select the Bug report template to get started.
|
||||
* Select the Bug Report template to get started.
|
||||
* **Use a clear and descriptive title** for the issue to identify the problem.
|
||||
* **Describe the exact steps which reproduce the problem** in as many details as possible. Trace the steps you took and **don't just say what you did, but explain how you did it**.
|
||||
* **Include additional data** in the issue. This encompasses your operating system, the version of Wabbajack and your log file.
|
||||
* **Include additional data** in the issue. This encompasses your operating system, the version of Wabbajack that was used and your log file.
|
||||
* **Upload the stacktrace or your entire log file** to the issue using the [Code Highlighting](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#code) feature of Markdown.
|
||||
|
||||
### Suggesting Enhancements
|
||||
|
||||
Enhancements can be everything from fixing typos to a complete revamp of documents in the repo. You can just use Github for making changes by clicking the pencil icon in the top right corner of a file.
|
||||
Enhancements can be everything from fixing typos to a complete revamp of documents in the repo. You can just use GitHub for making changes by clicking the pencil icon in the top right corner of a file.
|
||||
|
||||
### Code Contribution
|
||||
|
||||
This is where the fun begins. Wabbajack is programmed in C# so having a decent amount of knowledge in that language or in C/C++ is good to have. You also want to make sure that you have a basic understanding of the git workflow.
|
||||
This is where the fun begins. Wabbajack is programmed in C# so having a decent amount of knowledge in that language or in C/C++ is good to have. You also want to make sure that you have a basic understanding of the Git workflow.
|
||||
|
||||
#### Visual Studio 2019
|
||||
|
||||
You can download it [here](https://visualstudio.microsoft.com/vs/) but make sure to select the Community Edition as the other ones come at a cost. When installing VS you will be prompted to select a Workload and components. You will need:
|
||||
You can download it [here](https://visualstudio.microsoft.com/vs/) but make sure to select the Community Edition as the other ones come at a cost. When installing Visual Studio you will be prompted to select a Workload and components. You will need the following:
|
||||
|
||||
* **.NET desktop development** from the Workload tab
|
||||
* **.NET Framework 4.7.2 SDK and targeting pack** from the .NET section
|
||||
* **NuGet package manager** from the Code tools section
|
||||
* **NuGet package manager** from the Code Tools section
|
||||
* **C# and Visual Basic** from the Development activities
|
||||
|
||||
The installer may have selected other options as well but these are the most important ones.
|
||||
|
||||
### Starting development
|
||||
### Starting Development
|
||||
|
||||
1) **Fork and clone the project:** go to the Github repo page, click the fork button, copy the url from the forked repo, navigate to your project folder, open Git Bash or normal command prompt and type `git clone url name` and replace url with the copied url and name with the folder name
|
||||
2) **Open Wabbajack.sln** in Visual Studio 2019
|
||||
3) **Download NuGet Packages** by selecting the solution and *Right Click*->*Restore NuGet Packages*
|
||||
1) **Fork and clone the project:** go to the GitHub repo page, click the fork button, copy the url from the forked repo, navigate to your project folder, open Git Bash or normal command prompt and type `git clone url name` and replace url with the copied URL and name with the folder name.
|
||||
2) **Open Wabbajack.sln** in Visual Studio 2019.
|
||||
3) **Download NuGet Packages** by selecting the solution and *Right Click* -> *Restore NuGet Packages*.
|
||||
|
||||
It may take a while for Visual Studio to download all packages and update all References so be patience. Once all packages are downloaded go and try building Wabbajack. If the build is successful than good job, if not head over to the *#wabbajack-development* channel on the discord and talk about your build error.
|
||||
|
||||
@ -51,9 +51,9 @@ It may take a while for Visual Studio to download all packages and update all Re
|
||||
|
||||
As a C# project, you should follow the [C# Coding Style](https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/coding-style.md). Further more you should never submit commits to your *master* branch, even if it's just a fork. Create a new branch with a meaningful name or the name of your issue/request and commit to that.
|
||||
|
||||
You commits should also be elegant. Check [this](https://github.com/git-for-windows/git/wiki/Good-commits) post for good practices.
|
||||
Your commits should also be elegant. Check out [this](https://github.com/git-for-windows/git/wiki/Good-commits) post for good practices.
|
||||
|
||||
Updating your fork is important and easy. Open your terminal of choice inside the project folder and add the original repo as a new remote:
|
||||
Updating your fork is important and easy. Open your terminal of choice inside the project folder and add the original repository as a new remote:
|
||||
|
||||
`git remote add upstream https://github.com/wabbajack-tools/wabbajack.git`
|
||||
|
||||
@ -74,5 +74,6 @@ other branch:
|
||||
#### Submitting Code Changes
|
||||
|
||||
Before you go and open a pull request, make sure that your code actually runs. Build the project with your changes and test the application with its new features against your testing modlist. This testing modlist should be an MO2 installation with some mods installed that worked on the version without your changes and was not modified since then.
|
||||
Running the Unit Tests should also give you a good overview of what currently works.
|
||||
|
||||
If everything works as intended and you found no bug in testing, go ahead and open a pull request. Your request should contain information about why you want to change something, what you changed and how you did it.
|
||||
If everything works as intended and you found no bugs in testing, go ahead and open a pull request. Your request should contain information about why you want to change something, what you changed and how you did it.
|
||||
|
||||
@ -1,7 +0,0 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
|
||||
<configuration>
|
||||
<startup>
|
||||
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
|
||||
</startup>
|
||||
</configuration>
|
||||
180
Compression.BSA.Test/BSATests.cs
Normal file
180
Compression.BSA.Test/BSATests.cs
Normal file
@ -0,0 +1,180 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Threading.Tasks;
|
||||
using Alphaleonis.Win32.Filesystem;
|
||||
using Microsoft.VisualStudio.TestTools.UnitTesting;
|
||||
using Newtonsoft.Json;
|
||||
using Wabbajack.Common;
|
||||
using Wabbajack.Common.CSP;
|
||||
using Wabbajack.Lib.Downloaders;
|
||||
using Wabbajack.Lib.NexusApi;
|
||||
using Directory = Alphaleonis.Win32.Filesystem.Directory;
|
||||
using File = Alphaleonis.Win32.Filesystem.File;
|
||||
using FileInfo = Alphaleonis.Win32.Filesystem.FileInfo;
|
||||
using Path = Alphaleonis.Win32.Filesystem.Path;
|
||||
|
||||
namespace Compression.BSA.Test
|
||||
{
|
||||
[TestClass]
|
||||
public class BSATests
|
||||
{
|
||||
private static string StagingFolder = "NexusDownloads";
|
||||
private static string BSAFolder = "BSAs";
|
||||
private static string TestDir = "BSA Test Dir";
|
||||
private static string TempDir = "BSA Temp Dir";
|
||||
|
||||
public TestContext TestContext { get; set; }
|
||||
|
||||
[ClassInitialize]
|
||||
public static async Task Setup(TestContext TestContext)
|
||||
{
|
||||
|
||||
Utils.LogMessages.Subscribe(f => TestContext.WriteLine(f));
|
||||
if (!Directory.Exists(StagingFolder))
|
||||
Directory.CreateDirectory(StagingFolder);
|
||||
|
||||
if (!Directory.Exists(BSAFolder))
|
||||
Directory.CreateDirectory(BSAFolder);
|
||||
|
||||
var mod_ids = new[]
|
||||
{
|
||||
(Game.SkyrimSpecialEdition, 12604), // SkyUI
|
||||
(Game.Skyrim, 3863), // SkyUI
|
||||
(Game.Skyrim, 51473), // iNeed
|
||||
(Game.Fallout4, 22223) // 10mm SMG
|
||||
};
|
||||
|
||||
foreach (var info in mod_ids)
|
||||
{
|
||||
var filename = DownloadMod(info);
|
||||
var folder = Path.Combine(BSAFolder, info.Item1.ToString(), info.Item2.ToString());
|
||||
if (!Directory.Exists(folder))
|
||||
Directory.CreateDirectory(folder);
|
||||
await FileExtractor.ExtractAll(filename, folder);
|
||||
}
|
||||
}
|
||||
|
||||
private static string DownloadMod((Game, int) info)
|
||||
{
|
||||
using (var client = new NexusApiClient())
|
||||
{
|
||||
var results = client.GetModFiles(info.Item1, info.Item2);
|
||||
var file = results.FirstOrDefault(f => f.is_primary) ??
|
||||
results.OrderByDescending(f => f.uploaded_timestamp).First();
|
||||
var src = Path.Combine(StagingFolder, file.file_name);
|
||||
|
||||
if (File.Exists(src)) return src;
|
||||
|
||||
var state = new NexusDownloader.State
|
||||
{
|
||||
ModID = info.Item2.ToString(),
|
||||
GameName = GameRegistry.Games[info.Item1].NexusName,
|
||||
FileID = file.file_id.ToString()
|
||||
};
|
||||
state.Download(src);
|
||||
return src;
|
||||
}
|
||||
}
|
||||
|
||||
public static IEnumerable<object[]> BSAs()
|
||||
{
|
||||
return Directory.EnumerateFiles(BSAFolder, "*", DirectoryEnumerationOptions.Recursive)
|
||||
.Where(f => Consts.SupportedBSAs.Contains(Path.GetExtension(f)))
|
||||
.Select(nm => new object[] {nm});
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
[DataTestMethod]
|
||||
[DynamicData(nameof(BSAs), DynamicDataSourceType.Method)]
|
||||
public async Task BSACompressionRecompression(string bsa)
|
||||
{
|
||||
TestContext.WriteLine($"From {bsa}");
|
||||
TestContext.WriteLine("Cleaning Output Dir");
|
||||
if (Directory.Exists(TempDir)) Directory.Delete(TempDir, true);
|
||||
//if (Directory.Exists(ArchiveTempDir)) Directory.Delete(ArchiveTempDir, true);
|
||||
Directory.CreateDirectory(TempDir);
|
||||
|
||||
TestContext.WriteLine($"Reading {bsa}");
|
||||
using (var a = await BSADispatch.OpenRead(bsa))
|
||||
{
|
||||
await a.Files.UnorderedParallelDo(async file =>
|
||||
{
|
||||
var abs_name = Path.Combine(TempDir, file.Path);
|
||||
ViaJson(file.State);
|
||||
|
||||
if (!Directory.Exists(Path.GetDirectoryName(abs_name)))
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(abs_name));
|
||||
|
||||
|
||||
using (var fs = File.OpenWrite(abs_name))
|
||||
{
|
||||
await file.CopyDataToAsync(fs);
|
||||
}
|
||||
|
||||
Assert.AreEqual(file.Size, new FileInfo(abs_name).Length);
|
||||
});
|
||||
|
||||
Console.WriteLine($"Building {bsa}");
|
||||
string TempFile = Path.Combine("tmp.bsa");
|
||||
|
||||
using (var w = ViaJson(a.State).MakeBuilder())
|
||||
{
|
||||
await a.Files.UnorderedParallelDo(async file =>
|
||||
{
|
||||
var abs_path = Path.Combine(TempDir, file.Path);
|
||||
using (var str = File.OpenRead(abs_path))
|
||||
{
|
||||
await w.AddFile(ViaJson(file.State), str);
|
||||
}
|
||||
});
|
||||
await w.Build(TempFile);
|
||||
}
|
||||
|
||||
Console.WriteLine($"Verifying {bsa}");
|
||||
using (var b = await BSADispatch.OpenRead(TempFile))
|
||||
{
|
||||
|
||||
Console.WriteLine($"Performing A/B tests on {bsa}");
|
||||
Assert.AreEqual(JsonConvert.SerializeObject(a.State), JsonConvert.SerializeObject(b.State));
|
||||
|
||||
// Check same number of files
|
||||
Assert.AreEqual(a.Files.Count(), b.Files.Count());
|
||||
var idx = 0;
|
||||
|
||||
await a.Files.Zip(b.Files, (ai, bi) => (ai, bi))
|
||||
.UnorderedParallelDo(async pair =>
|
||||
{
|
||||
idx++;
|
||||
Assert.AreEqual(JsonConvert.SerializeObject(pair.ai.State),
|
||||
JsonConvert.SerializeObject(pair.bi.State));
|
||||
//Console.WriteLine($" - {pair.ai.Path}");
|
||||
Assert.AreEqual(pair.ai.Path, pair.bi.Path);
|
||||
//Equal(pair.ai.Compressed, pair.bi.Compressed);
|
||||
Assert.AreEqual(pair.ai.Size, pair.bi.Size);
|
||||
CollectionAssert.AreEqual(await GetData(pair.ai), await GetData(pair.bi));
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task<byte[]> GetData(IFile pairAi)
|
||||
{
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
await pairAi.CopyDataToAsync(ms);
|
||||
return ms.ToArray();
|
||||
}
|
||||
}
|
||||
|
||||
public static T ViaJson<T>(T i)
|
||||
{
|
||||
var settings = new JsonSerializerSettings
|
||||
{
|
||||
TypeNameHandling = TypeNameHandling.All
|
||||
};
|
||||
return JsonConvert.DeserializeObject<T>(JsonConvert.SerializeObject(i, settings), settings);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,20 +1,26 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<Project ToolsVersion="15.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
|
||||
<PropertyGroup>
|
||||
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
|
||||
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
|
||||
<ProjectGuid>{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}</ProjectGuid>
|
||||
<OutputType>Exe</OutputType>
|
||||
<ProjectGuid>{9C004392-571A-4D28-A9F6-0E25115E6727}</ProjectGuid>
|
||||
<OutputType>Library</OutputType>
|
||||
<AppDesignerFolder>Properties</AppDesignerFolder>
|
||||
<RootNamespace>Compression.BSA.Test</RootNamespace>
|
||||
<AssemblyName>Compression.BSA.Test</AssemblyName>
|
||||
<TargetFrameworkVersion>v4.7.2</TargetFrameworkVersion>
|
||||
<FileAlignment>512</FileAlignment>
|
||||
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
|
||||
<Deterministic>true</Deterministic>
|
||||
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
|
||||
<VisualStudioVersion Condition="'$(VisualStudioVersion)' == ''">15.0</VisualStudioVersion>
|
||||
<VSToolsPath Condition="'$(VSToolsPath)' == ''">$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)</VSToolsPath>
|
||||
<ReferencePath>$(ProgramFiles)\Common Files\microsoft shared\VSTT\$(VisualStudioVersion)\UITestExtensionPackages</ReferencePath>
|
||||
<IsCodedUITest>False</IsCodedUITest>
|
||||
<TestProjectType>UnitTest</TestProjectType>
|
||||
<NuGetPackageImportStamp>
|
||||
</NuGetPackageImportStamp>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
|
||||
<PlatformTarget>x64</PlatformTarget>
|
||||
<DebugSymbols>true</DebugSymbols>
|
||||
<DebugType>full</DebugType>
|
||||
<Optimize>false</Optimize>
|
||||
@ -24,7 +30,6 @@
|
||||
<WarningLevel>4</WarningLevel>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
|
||||
<PlatformTarget>AnyCPU</PlatformTarget>
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<Optimize>true</Optimize>
|
||||
<OutputPath>bin\Release\</OutputPath>
|
||||
@ -38,9 +43,9 @@
|
||||
<DefineConstants>DEBUG;TRACE</DefineConstants>
|
||||
<DebugType>full</DebugType>
|
||||
<PlatformTarget>x64</PlatformTarget>
|
||||
<LangVersion>7.3</LangVersion>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
<Prefer32Bit>true</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
|
||||
<OutputPath>bin\x64\Release\</OutputPath>
|
||||
@ -48,62 +53,48 @@
|
||||
<Optimize>true</Optimize>
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<PlatformTarget>x64</PlatformTarget>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
<Prefer32Bit>true</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
|
||||
<DebugSymbols>true</DebugSymbols>
|
||||
<OutputPath>bin\x86\Debug\</OutputPath>
|
||||
<DefineConstants>DEBUG;TRACE</DefineConstants>
|
||||
<DebugType>full</DebugType>
|
||||
<PlatformTarget>x86</PlatformTarget>
|
||||
<LangVersion>7.3</LangVersion>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
<Prefer32Bit>true</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
|
||||
<OutputPath>bin\x86\Release\</OutputPath>
|
||||
<DefineConstants>TRACE</DefineConstants>
|
||||
<Optimize>true</Optimize>
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<PlatformTarget>x86</PlatformTarget>
|
||||
<LangVersion>7.3</LangVersion>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
<Prefer32Bit>true</Prefer32Bit>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Reference Include="System" />
|
||||
<Reference Include="System.Core" />
|
||||
<Reference Include="System.Xml.Linq" />
|
||||
<Reference Include="System.Data.DataSetExtensions" />
|
||||
<Reference Include="Microsoft.CSharp" />
|
||||
<Reference Include="System.Data" />
|
||||
<Reference Include="System.Net.Http" />
|
||||
<Reference Include="System.Xml" />
|
||||
<Reference Include="System.Transactions" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Compile Include="Program.cs" />
|
||||
<Compile Include="BSATests.cs" />
|
||||
<Compile Include="Properties\AssemblyInfo.cs" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<None Include="App.config" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\Compression.BSA\Compression.BSA.csproj">
|
||||
<Project>{ff5d892f-8ff4-44fc-8f7f-cd58f307ad1b}</Project>
|
||||
<Name>Compression.BSA</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Common.CSP\Wabbajack.Common.CSP.csproj">
|
||||
<Project>{9e69bc98-1512-4977-b683-6e7e5292c0b8}</Project>
|
||||
<Name>Wabbajack.Common.CSP</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Common\Wabbajack.Common.csproj">
|
||||
<Project>{b3f3fb6e-b9eb-4f49-9875-d78578bc7ae5}</Project>
|
||||
<Name>Wabbajack.Common</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Lib\Wabbajack.Lib.csproj">
|
||||
<Project>{0a820830-a298-497d-85e0-e9a89efef5fe}</Project>
|
||||
<Name>Wabbajack.Lib</Name>
|
||||
</ProjectReference>
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Newtonsoft.Json">
|
||||
<Version>12.0.2</Version>
|
||||
<PackageReference Include="AlphaFS">
|
||||
<Version>2.2.6</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="SharpZipLib">
|
||||
<Version>1.2.0</Version>
|
||||
<PackageReference Include="MSTest.TestAdapter">
|
||||
<Version>1.3.2</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="MSTest.TestFramework">
|
||||
<Version>1.3.2</Version>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
<Import Project="$(VSToolsPath)\TeamTest\Microsoft.TestTools.targets" Condition="Exists('$(VSToolsPath)\TeamTest\Microsoft.TestTools.targets')" />
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
|
||||
</Project>
|
||||
@ -1,205 +0,0 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Threading.Tasks;
|
||||
using Newtonsoft.Json;
|
||||
|
||||
namespace Compression.BSA.Test
|
||||
{
|
||||
internal class Program
|
||||
{
|
||||
//private const string TestDirBSA = @"D:\MO2 Instances\F4EE";
|
||||
//private const string TestDirBA2 = @"D:\MO2 Instances\F4EE";
|
||||
private const string TestDir = @"D:\MO2 Instances";
|
||||
//private const string TestDir = @"D:\Steam\steamapps\common\Fallout 4";
|
||||
private const string TempDir = @"c:\tmp\out\f4ee";
|
||||
private const string ArchiveTempDir = @"c:\tmp\out\archive";
|
||||
|
||||
//private const string Archive2Location = @"D:\Steam\steamapps\common\Fallout 4\Tools\Archive2\Archive2.exe";
|
||||
|
||||
private static void Main(string[] args)
|
||||
{
|
||||
foreach (var bsa in Directory.EnumerateFiles(TestDir, "*.ba2", SearchOption.AllDirectories)
|
||||
//.Concat(Directory.EnumerateFiles(TestDir, "*.bsa", SearchOption.AllDirectories))
|
||||
)
|
||||
{
|
||||
Console.WriteLine($"From {bsa}");
|
||||
Console.WriteLine("Cleaning Output Dir");
|
||||
if (Directory.Exists(TempDir)) Directory.Delete(TempDir, true);
|
||||
if (Directory.Exists(ArchiveTempDir)) Directory.Delete(ArchiveTempDir, true);
|
||||
Directory.CreateDirectory(TempDir);
|
||||
|
||||
Console.WriteLine($"Reading {bsa}");
|
||||
using (var a = BSADispatch.OpenRead(bsa))
|
||||
{
|
||||
Parallel.ForEach(a.Files, file =>
|
||||
{
|
||||
var abs_name = Path.Combine(TempDir, file.Path);
|
||||
ViaJson(file.State);
|
||||
|
||||
if (!Directory.Exists(Path.GetDirectoryName(abs_name)))
|
||||
Directory.CreateDirectory(Path.GetDirectoryName(abs_name));
|
||||
|
||||
|
||||
using (var fs = File.OpenWrite(abs_name))
|
||||
{
|
||||
file.CopyDataTo(fs);
|
||||
}
|
||||
|
||||
|
||||
Equal(file.Size, new FileInfo(abs_name).Length);
|
||||
|
||||
});
|
||||
|
||||
/*
|
||||
Console.WriteLine("Extracting via Archive.exe");
|
||||
if (bsa.ToLower().EndsWith(".ba2"))
|
||||
{
|
||||
var p = Process.Start(Archive2Location, $"\"{bsa}\" -e=\"{ArchiveTempDir}\"");
|
||||
p.WaitForExit();
|
||||
|
||||
foreach (var file in a.Files)
|
||||
{
|
||||
var a_path = Path.Combine(TempDir, file.Path);
|
||||
var b_path = Path.Combine(ArchiveTempDir, file.Path);
|
||||
Equal(new FileInfo(a_path).Length, new FileInfo(b_path).Length);
|
||||
Equal(File.ReadAllBytes(a_path), File.ReadAllBytes(b_path));
|
||||
}
|
||||
}*/
|
||||
|
||||
|
||||
Console.WriteLine($"Building {bsa}");
|
||||
|
||||
using (var w = ViaJson(a.State).MakeBuilder())
|
||||
{
|
||||
|
||||
Parallel.ForEach(a.Files, file =>
|
||||
{
|
||||
var abs_path = Path.Combine(TempDir, file.Path);
|
||||
using (var str = File.OpenRead(abs_path))
|
||||
{
|
||||
w.AddFile(ViaJson(file.State), str);
|
||||
}
|
||||
});
|
||||
|
||||
w.Build("c:\\tmp\\tmp.bsa");
|
||||
}
|
||||
|
||||
Console.WriteLine($"Verifying {bsa}");
|
||||
using (var b = BSADispatch.OpenRead("c:\\tmp\\tmp.bsa"))
|
||||
{
|
||||
|
||||
Console.WriteLine($"Performing A/B tests on {bsa}");
|
||||
Equal(JsonConvert.SerializeObject(a.State), JsonConvert.SerializeObject(b.State));
|
||||
|
||||
//Equal((uint) a.ArchiveFlags, (uint) b.ArchiveFlags);
|
||||
//Equal((uint) a.FileFlags, (uint) b.FileFlags);
|
||||
|
||||
// Check same number of files
|
||||
Equal(a.Files.Count(), b.Files.Count());
|
||||
var idx = 0;
|
||||
foreach (var pair in a.Files.Zip(b.Files, (ai, bi) => (ai, bi)))
|
||||
{
|
||||
idx++;
|
||||
Equal(JsonConvert.SerializeObject(pair.ai.State),
|
||||
JsonConvert.SerializeObject(pair.bi.State));
|
||||
//Console.WriteLine($" - {pair.ai.Path}");
|
||||
Equal(pair.ai.Path, pair.bi.Path);
|
||||
//Equal(pair.ai.Compressed, pair.bi.Compressed);
|
||||
Equal(pair.ai.Size, pair.bi.Size);
|
||||
Equal(GetData(pair.ai), GetData(pair.bi));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static byte[] GetData(IFile pairAi)
|
||||
{
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
pairAi.CopyDataTo(ms);
|
||||
return ms.ToArray();
|
||||
}
|
||||
}
|
||||
|
||||
public static T ViaJson<T>(T i)
|
||||
{
|
||||
var settings = new JsonSerializerSettings
|
||||
{
|
||||
TypeNameHandling = TypeNameHandling.All
|
||||
};
|
||||
return JsonConvert.DeserializeObject<T>(JsonConvert.SerializeObject(i, settings), settings);
|
||||
}
|
||||
|
||||
private static void Equal(HashSet<string> a, HashSet<string> b)
|
||||
{
|
||||
Equal(a.Count, b.Count);
|
||||
|
||||
foreach (var itm in a)
|
||||
Equal(b.Contains(itm));
|
||||
}
|
||||
|
||||
private static void Equal(bool v)
|
||||
{
|
||||
if (!v) throw new InvalidDataException("False");
|
||||
}
|
||||
|
||||
public static void Equal(uint a, uint b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(long a, long b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(ulong a, ulong b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(int a, int b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(string a, string b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(bool a, bool b)
|
||||
{
|
||||
if (a == b) return;
|
||||
|
||||
throw new InvalidDataException($"{a} != {b}");
|
||||
}
|
||||
|
||||
public static void Equal(byte[] a, byte[] b)
|
||||
{
|
||||
if (a.Length != b.Length) throw new InvalidDataException("Byte array sizes are not equal");
|
||||
|
||||
for (var idx = 0; idx < a.Length; idx++)
|
||||
{
|
||||
if (a[idx] != b[idx])
|
||||
{
|
||||
Console.WriteLine($"Byte array contents not equal at {idx} - {a[idx]} vs {b[idx]}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,9 +1,7 @@
|
||||
using System.Reflection;
|
||||
using System.Reflection;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
// General Information about an assembly is controlled through the following
|
||||
// set of attributes. Change these attribute values to modify the information
|
||||
// associated with an assembly.
|
||||
[assembly: AssemblyTitle("Compression.BSA.Test")]
|
||||
[assembly: AssemblyDescription("")]
|
||||
[assembly: AssemblyConfiguration("")]
|
||||
@ -13,23 +11,10 @@ using System.Runtime.InteropServices;
|
||||
[assembly: AssemblyTrademark("")]
|
||||
[assembly: AssemblyCulture("")]
|
||||
|
||||
// Setting ComVisible to false makes the types in this assembly not visible
|
||||
// to COM components. If you need to access a type in this assembly from
|
||||
// COM, set the ComVisible attribute to true on that type.
|
||||
[assembly: ComVisible(false)]
|
||||
|
||||
// The following GUID is for the ID of the typelib if this project is exposed to COM
|
||||
[assembly: Guid("ba2cfea1-072b-42d6-822a-8c6d0e3ae5d9")]
|
||||
[assembly: Guid("9c004392-571a-4d28-a9f6-0e25115e6727")]
|
||||
|
||||
// Version information for an assembly consists of the following four values:
|
||||
//
|
||||
// Major Version
|
||||
// Minor Version
|
||||
// Build Number
|
||||
// Revision
|
||||
//
|
||||
// You can specify all the values or you can default the Build and Revision Numbers
|
||||
// by using the '*' as shown below:
|
||||
// [assembly: AssemblyVersion("1.0.*")]
|
||||
[assembly: AssemblyVersion("1.0.0.0")]
|
||||
[assembly: AssemblyFileVersion("1.0.0.0")]
|
||||
[assembly: AssemblyFileVersion("1.0.0.0")]
|
||||
|
||||
@ -1,11 +1,13 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Dynamic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using ICSharpCode.SharpZipLib.Zip.Compression;
|
||||
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
|
||||
using Wabbajack.Common.CSP;
|
||||
|
||||
namespace Compression.BSA
|
||||
{
|
||||
@ -17,7 +19,7 @@ namespace Compression.BSA
|
||||
|
||||
int Index { get; }
|
||||
|
||||
void WriteData(BinaryWriter wtr);
|
||||
Task WriteData(BinaryWriter wtr);
|
||||
void WriteHeader(BinaryWriter wtr);
|
||||
|
||||
}
|
||||
@ -35,22 +37,22 @@ namespace Compression.BSA
|
||||
{
|
||||
}
|
||||
|
||||
public void AddFile(FileStateObject state, Stream src)
|
||||
public async Task AddFile(FileStateObject state, Stream src)
|
||||
{
|
||||
switch (_state.Type)
|
||||
{
|
||||
case EntryType.GNRL:
|
||||
var result = new BA2FileEntryBuilder((BA2FileEntryState)state, src);
|
||||
var result = await BA2FileEntryBuilder.Create((BA2FileEntryState)state, src);
|
||||
lock(_entries) _entries.Add(result);
|
||||
break;
|
||||
case EntryType.DX10:
|
||||
var resultdx10 = new BA2DX10FileEntryBuilder((BA2DX10EntryState)state, src);
|
||||
var resultdx10 = await BA2DX10FileEntryBuilder.Create((BA2DX10EntryState)state, src);
|
||||
lock(_entries) _entries.Add(resultdx10);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
public void Build(string filename)
|
||||
public async Task Build(string filename)
|
||||
{
|
||||
SortEntries();
|
||||
using (var fs = File.OpenWrite(filename))
|
||||
@ -70,7 +72,7 @@ namespace Compression.BSA
|
||||
|
||||
foreach (var entry in _entries)
|
||||
{
|
||||
entry.WriteData(bw);
|
||||
await entry.WriteData(bw);
|
||||
}
|
||||
|
||||
if (_state.HasNameTable)
|
||||
@ -84,7 +86,7 @@ namespace Compression.BSA
|
||||
{
|
||||
var bytes = Encoding.UTF7.GetBytes(entry.FullName);
|
||||
bw.Write((ushort)bytes.Length);
|
||||
bw.Write(bytes);
|
||||
await bw.BaseStream.WriteAsync(bytes, 0, bytes.Length);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -101,14 +103,21 @@ namespace Compression.BSA
|
||||
private BA2DX10EntryState _state;
|
||||
private List<ChunkBuilder> _chunks;
|
||||
|
||||
public BA2DX10FileEntryBuilder(BA2DX10EntryState state, Stream src)
|
||||
public static async Task<BA2DX10FileEntryBuilder> Create(BA2DX10EntryState state, Stream src)
|
||||
{
|
||||
_state = state;
|
||||
var builder = new BA2DX10FileEntryBuilder();
|
||||
builder._state = state;
|
||||
|
||||
var header_size = DDS.HeaderSizeForFormat((DXGI_FORMAT) state.PixelFormat) + 4;
|
||||
new BinaryReader(src).ReadBytes((int)header_size);
|
||||
|
||||
_chunks = _state.Chunks.Select(ch => new ChunkBuilder(state, ch, src)).ToList();
|
||||
// This can't be parallel because it all runs off the same base IO stream.
|
||||
builder._chunks = new List<ChunkBuilder>();
|
||||
|
||||
foreach (var chunk in state.Chunks)
|
||||
builder._chunks.Add(await ChunkBuilder.Create(state, chunk, src));
|
||||
|
||||
return builder;
|
||||
}
|
||||
|
||||
public uint FileHash => _state.NameHash;
|
||||
@ -134,10 +143,10 @@ namespace Compression.BSA
|
||||
chunk.WriteHeader(bw);
|
||||
}
|
||||
|
||||
public void WriteData(BinaryWriter wtr)
|
||||
public async Task WriteData(BinaryWriter wtr)
|
||||
{
|
||||
foreach (var chunk in _chunks)
|
||||
chunk.WriteData(wtr);
|
||||
await chunk.WriteData(wtr);
|
||||
}
|
||||
|
||||
}
|
||||
@ -149,28 +158,31 @@ namespace Compression.BSA
|
||||
private uint _packSize;
|
||||
private long _offsetOffset;
|
||||
|
||||
public ChunkBuilder(BA2DX10EntryState state, ChunkState ch, Stream src)
|
||||
public static async Task<ChunkBuilder> Create(BA2DX10EntryState state, ChunkState chunk, Stream src)
|
||||
{
|
||||
_chunk = ch;
|
||||
var builder = new ChunkBuilder {_chunk = chunk};
|
||||
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
src.CopyToLimit(ms, (int)_chunk.FullSz);
|
||||
_data = ms.ToArray();
|
||||
await src.CopyToLimitAsync(ms, (int)chunk.FullSz);
|
||||
builder._data = ms.ToArray();
|
||||
}
|
||||
|
||||
if (_chunk.Compressed)
|
||||
if (!chunk.Compressed) return builder;
|
||||
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
using (var ms = new MemoryStream())
|
||||
using (var ds = new DeflaterOutputStream(ms))
|
||||
{
|
||||
using (var ds = new DeflaterOutputStream(ms))
|
||||
{
|
||||
ds.Write(_data, 0, _data.Length);
|
||||
}
|
||||
_data = ms.ToArray();
|
||||
ds.Write(builder._data, 0, builder._data.Length);
|
||||
}
|
||||
_packSize = (uint)_data.Length;
|
||||
|
||||
builder._data = ms.ToArray();
|
||||
}
|
||||
|
||||
builder._packSize = (uint) builder._data.Length;
|
||||
|
||||
return builder;
|
||||
}
|
||||
|
||||
public void WriteHeader(BinaryWriter bw)
|
||||
@ -185,13 +197,13 @@ namespace Compression.BSA
|
||||
|
||||
}
|
||||
|
||||
public void WriteData(BinaryWriter bw)
|
||||
public async Task WriteData(BinaryWriter bw)
|
||||
{
|
||||
var pos = bw.BaseStream.Position;
|
||||
bw.BaseStream.Position = _offsetOffset;
|
||||
bw.Write((ulong)pos);
|
||||
bw.BaseStream.Position = pos;
|
||||
bw.Write(_data);
|
||||
await bw.BaseStream.WriteAsync(_data, 0, _data.Length);
|
||||
}
|
||||
}
|
||||
|
||||
@ -203,15 +215,17 @@ namespace Compression.BSA
|
||||
private BA2FileEntryState _state;
|
||||
private long _offsetOffset;
|
||||
|
||||
public BA2FileEntryBuilder(BA2FileEntryState state, Stream src)
|
||||
public static async Task<BA2FileEntryBuilder> Create(BA2FileEntryState state, Stream src)
|
||||
{
|
||||
_state = state;
|
||||
var builder = new BA2FileEntryBuilder();
|
||||
builder._state = state;
|
||||
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
src.CopyTo(ms);
|
||||
_data = ms.ToArray();
|
||||
await src.CopyToAsync(ms);
|
||||
builder._data = ms.ToArray();
|
||||
}
|
||||
_rawSize = _data.Length;
|
||||
builder._rawSize = builder._data.Length;
|
||||
|
||||
if (state.Compressed)
|
||||
{
|
||||
@ -219,14 +233,13 @@ namespace Compression.BSA
|
||||
{
|
||||
using (var ds = new DeflaterOutputStream(ms))
|
||||
{
|
||||
ds.Write(_data, 0, _data.Length);
|
||||
await ds.WriteAsync(builder._data, 0, builder._data.Length);
|
||||
}
|
||||
_data = ms.ToArray();
|
||||
builder._data = ms.ToArray();
|
||||
}
|
||||
|
||||
_size = _data.Length;
|
||||
builder._size = builder._data.Length;
|
||||
}
|
||||
|
||||
return builder;
|
||||
}
|
||||
|
||||
public uint FileHash => _state.NameHash;
|
||||
@ -247,13 +260,13 @@ namespace Compression.BSA
|
||||
wtr.Write(_state.Align);
|
||||
}
|
||||
|
||||
public void WriteData(BinaryWriter wtr)
|
||||
public async Task WriteData(BinaryWriter wtr)
|
||||
{
|
||||
var pos = wtr.BaseStream.Position;
|
||||
wtr.BaseStream.Seek(_offsetOffset, SeekOrigin.Begin);
|
||||
wtr.Write((ulong)pos);
|
||||
wtr.BaseStream.Position = pos;
|
||||
wtr.Write(_data);
|
||||
await wtr.BaseStream.WriteAsync(_data, 0, _data.Length);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -188,7 +188,7 @@ namespace Compression.BSA
|
||||
|
||||
public uint HeaderSize => DDS.HeaderSizeForFormat((DXGI_FORMAT)_format);
|
||||
|
||||
public void CopyDataTo(Stream output)
|
||||
public async Task CopyDataToAsync(Stream output)
|
||||
{
|
||||
var bw = new BinaryWriter(output);
|
||||
|
||||
@ -199,25 +199,25 @@ namespace Compression.BSA
|
||||
{
|
||||
foreach (var chunk in _chunks)
|
||||
{
|
||||
byte[] full = new byte[chunk._fullSz];
|
||||
var full = new byte[chunk._fullSz];
|
||||
var isCompressed = chunk._packSz != 0;
|
||||
|
||||
br.BaseStream.Seek((long)chunk._offset, SeekOrigin.Begin);
|
||||
|
||||
if (!isCompressed)
|
||||
{
|
||||
br.Read(full, 0, full.Length);
|
||||
await br.BaseStream.ReadAsync(full, 0, full.Length);
|
||||
}
|
||||
else
|
||||
{
|
||||
byte[] compressed = new byte[chunk._packSz];
|
||||
br.Read(compressed, 0, compressed.Length);
|
||||
await br.BaseStream.ReadAsync(compressed, 0, compressed.Length);
|
||||
var inflater = new Inflater();
|
||||
inflater.SetInput(compressed);
|
||||
inflater.Inflate(full);
|
||||
}
|
||||
|
||||
bw.Write(full);
|
||||
await bw.BaseStream.WriteAsync(full, 0, full.Length);
|
||||
}
|
||||
}
|
||||
|
||||
@ -450,21 +450,19 @@ namespace Compression.BSA
|
||||
public uint Size => _realSize;
|
||||
public FileStateObject State => new BA2FileEntryState(this);
|
||||
|
||||
public void CopyDataTo(Stream output)
|
||||
public async Task CopyDataToAsync(Stream output)
|
||||
{
|
||||
using (var bw = new BinaryWriter(output))
|
||||
using (var fs = File.OpenRead(_bsa._filename))
|
||||
using (var br = new BinaryReader(fs))
|
||||
{
|
||||
br.BaseStream.Seek((long) _offset, SeekOrigin.Begin);
|
||||
fs.Seek((long) _offset, SeekOrigin.Begin);
|
||||
uint len = Compressed ? _size : _realSize;
|
||||
|
||||
var bytes = new byte[len];
|
||||
br.Read(bytes, 0, (int) len);
|
||||
await fs.ReadAsync(bytes, 0, (int) len);
|
||||
|
||||
if (!Compressed)
|
||||
{
|
||||
bw.Write(bytes);
|
||||
await output.WriteAsync(bytes, 0, bytes.Length);
|
||||
}
|
||||
else
|
||||
{
|
||||
@ -472,7 +470,7 @@ namespace Compression.BSA
|
||||
var inflater = new Inflater();
|
||||
inflater.SetInput(bytes);
|
||||
inflater.Inflate(uncompressed);
|
||||
bw.Write(uncompressed);
|
||||
await output.WriteAsync(uncompressed, 0, uncompressed.Length);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -3,6 +3,7 @@ using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
|
||||
using K4os.Compression.LZ4;
|
||||
using K4os.Compression.LZ4.Streams;
|
||||
@ -76,24 +77,11 @@ namespace Compression.BSA
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
|
||||
public FileEntry AddFile(string path, Stream src, bool flipCompression = false)
|
||||
{
|
||||
var r = new FileEntry(this, path, src, flipCompression);
|
||||
|
||||
lock (this)
|
||||
{
|
||||
_files.Add(r);
|
||||
}
|
||||
|
||||
return r;
|
||||
}
|
||||
|
||||
public void AddFile(FileStateObject state, Stream src)
|
||||
public async Task AddFile(FileStateObject state, Stream src)
|
||||
{
|
||||
var ostate = (BSAFileStateObject) state;
|
||||
|
||||
var r = new FileEntry(this, ostate.Path, src, ostate.FlipCompression);
|
||||
var r = await FileEntry.Create(this, ostate.Path, src, ostate.FlipCompression);
|
||||
|
||||
lock (this)
|
||||
{
|
||||
@ -101,7 +89,7 @@ namespace Compression.BSA
|
||||
}
|
||||
}
|
||||
|
||||
public void Build(string outputName)
|
||||
public async Task Build(string outputName)
|
||||
{
|
||||
RegenFolderRecords();
|
||||
if (File.Exists(outputName)) File.Delete(outputName);
|
||||
@ -133,7 +121,8 @@ namespace Compression.BSA
|
||||
|
||||
foreach (var file in _files) wtr.Write(file._nameBytes);
|
||||
|
||||
foreach (var file in _files) file.WriteData(wtr);
|
||||
foreach (var file in _files)
|
||||
await file.WriteData(wtr);
|
||||
}
|
||||
}
|
||||
|
||||
@ -244,28 +233,30 @@ namespace Compression.BSA
|
||||
private long _offsetOffset;
|
||||
internal int _originalSize;
|
||||
internal string _path;
|
||||
private readonly byte[] _pathBSBytes;
|
||||
private byte[] _pathBSBytes;
|
||||
internal byte[] _pathBytes;
|
||||
internal byte[] _rawData;
|
||||
|
||||
public FileEntry(BSABuilder bsa, string path, Stream src, bool flipCompression)
|
||||
public static async Task<FileEntry> Create(BSABuilder bsa, string path, Stream src, bool flipCompression)
|
||||
{
|
||||
_bsa = bsa;
|
||||
_path = path.ToLowerInvariant();
|
||||
_name = System.IO.Path.GetFileName(_path);
|
||||
_hash = _name.GetBSAHash();
|
||||
_nameBytes = _name.ToTermString(bsa.HeaderType);
|
||||
_pathBytes = _path.ToTermString(bsa.HeaderType);
|
||||
_pathBSBytes = _path.ToBSString();
|
||||
_flipCompression = flipCompression;
|
||||
var entry = new FileEntry();
|
||||
entry._bsa = bsa;
|
||||
entry._path = path.ToLowerInvariant();
|
||||
entry._name = System.IO.Path.GetFileName(entry._path);
|
||||
entry._hash = entry._name.GetBSAHash();
|
||||
entry._nameBytes = entry._name.ToTermString(bsa.HeaderType);
|
||||
entry._pathBytes = entry._path.ToTermString(bsa.HeaderType);
|
||||
entry._pathBSBytes = entry._path.ToBSString();
|
||||
entry._flipCompression = flipCompression;
|
||||
|
||||
var ms = new MemoryStream();
|
||||
src.CopyTo(ms);
|
||||
_rawData = ms.ToArray();
|
||||
_originalSize = _rawData.Length;
|
||||
await src.CopyToAsync(ms);
|
||||
entry._rawData = ms.ToArray();
|
||||
entry._originalSize = entry._rawData.Length;
|
||||
|
||||
if (Compressed)
|
||||
CompressData();
|
||||
if (entry.Compressed)
|
||||
entry.CompressData();
|
||||
return entry;
|
||||
}
|
||||
|
||||
public bool Compressed
|
||||
@ -330,7 +321,7 @@ namespace Compression.BSA
|
||||
wtr.Write(0xDEADBEEF);
|
||||
}
|
||||
|
||||
internal void WriteData(BinaryWriter wtr)
|
||||
internal async Task WriteData(BinaryWriter wtr)
|
||||
{
|
||||
var offset = (uint) wtr.BaseStream.Position;
|
||||
wtr.BaseStream.Position = _offsetOffset;
|
||||
@ -342,11 +333,11 @@ namespace Compression.BSA
|
||||
if (Compressed)
|
||||
{
|
||||
wtr.Write((uint) _originalSize);
|
||||
wtr.Write(_rawData);
|
||||
await wtr.BaseStream.WriteAsync(_rawData, 0, _rawData.Length);
|
||||
}
|
||||
else
|
||||
{
|
||||
wtr.Write(_rawData);
|
||||
await wtr.BaseStream.WriteAsync(_rawData, 0, _rawData.Length);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -4,24 +4,29 @@ using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using Wabbajack.Common.CSP;
|
||||
|
||||
namespace Compression.BSA
|
||||
{
|
||||
public static class BSADispatch
|
||||
{
|
||||
public static IBSAReader OpenRead(string filename)
|
||||
public static Task<IBSAReader> OpenRead(string filename)
|
||||
{
|
||||
string fourcc = "";
|
||||
using (var file = File.OpenRead(filename))
|
||||
return CSPExtensions.ThreadedTask<IBSAReader>(() =>
|
||||
{
|
||||
fourcc = Encoding.ASCII.GetString(new BinaryReader(file).ReadBytes(4));
|
||||
}
|
||||
string fourcc = "";
|
||||
using (var file = File.OpenRead(filename))
|
||||
{
|
||||
fourcc = Encoding.ASCII.GetString(new BinaryReader(file).ReadBytes(4));
|
||||
}
|
||||
|
||||
if (fourcc == "BSA\0")
|
||||
return new BSAReader(filename);
|
||||
if (fourcc == "BTDX")
|
||||
return new BA2Reader(filename);
|
||||
throw new InvalidDataException("Filename is not a .bsa or .ba2, magic " + fourcc);
|
||||
if (fourcc == "BSA\0")
|
||||
return new BSAReader(filename);
|
||||
if (fourcc == "BTDX")
|
||||
return new BA2Reader(filename);
|
||||
throw new InvalidDataException("Filename is not a .bsa or .ba2, magic " + fourcc);
|
||||
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -3,6 +3,7 @@ using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
|
||||
using K4os.Compression.LZ4.Streams;
|
||||
using File = Alphaleonis.Win32.Filesystem.File;
|
||||
@ -302,7 +303,7 @@ namespace Compression.BSA
|
||||
_name = rdr.ReadStringTerm(_bsa.HeaderType);
|
||||
}
|
||||
|
||||
public void CopyDataTo(Stream output)
|
||||
public async Task CopyDataToAsync(Stream output)
|
||||
{
|
||||
using (var in_file = File.OpenRead(_bsa._fileName))
|
||||
using (var rdr = new BinaryReader(in_file))
|
||||
@ -314,11 +315,11 @@ namespace Compression.BSA
|
||||
if (Compressed)
|
||||
{
|
||||
var r = LZ4Stream.Decode(rdr.BaseStream);
|
||||
r.CopyToLimit(output, (int) _originalSize);
|
||||
await r.CopyToLimitAsync(output, (int) _originalSize);
|
||||
}
|
||||
else
|
||||
{
|
||||
rdr.BaseStream.CopyToLimit(output, (int) _onDiskSize);
|
||||
await rdr.BaseStream.CopyToLimitAsync(output, (int) _onDiskSize);
|
||||
}
|
||||
}
|
||||
else
|
||||
@ -326,18 +327,18 @@ namespace Compression.BSA
|
||||
if (Compressed)
|
||||
using (var z = new InflaterInputStream(rdr.BaseStream))
|
||||
{
|
||||
z.CopyToLimit(output, (int) _originalSize);
|
||||
await z.CopyToLimitAsync(output, (int) _originalSize);
|
||||
}
|
||||
else
|
||||
rdr.BaseStream.CopyToLimit(output, (int) _onDiskSize);
|
||||
await rdr.BaseStream.CopyToLimitAsync(output, (int) _onDiskSize);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public byte[] GetData()
|
||||
public async Task<byte[]> GetData()
|
||||
{
|
||||
var ms = new MemoryStream();
|
||||
CopyDataTo(ms);
|
||||
await CopyDataToAsync(ms);
|
||||
return ms.ToArray();
|
||||
}
|
||||
}
|
||||
|
||||
@ -110,5 +110,11 @@
|
||||
<Version>1.2.0</Version>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\Wabbajack.Common.CSP\Wabbajack.Common.CSP.csproj">
|
||||
<Project>{9e69bc98-1512-4977-b683-6e7e5292c0b8}</Project>
|
||||
<Name>Wabbajack.Common.CSP</Name>
|
||||
</ProjectReference>
|
||||
</ItemGroup>
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
|
||||
</Project>
|
||||
@ -19,8 +19,8 @@ namespace Compression.BSA
|
||||
|
||||
public interface IBSABuilder : IDisposable
|
||||
{
|
||||
void AddFile(FileStateObject state, Stream src);
|
||||
void Build(string filename);
|
||||
Task AddFile(FileStateObject state, Stream src);
|
||||
Task Build(string filename);
|
||||
}
|
||||
|
||||
public class ArchiveStateObject
|
||||
@ -59,6 +59,6 @@ namespace Compression.BSA
|
||||
/// in order to maintain thread-safe access.
|
||||
/// </summary>
|
||||
/// <param name="output"></param>
|
||||
void CopyDataTo(Stream output);
|
||||
Task CopyDataToAsync(Stream output);
|
||||
}
|
||||
}
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using Path = Alphaleonis.Win32.Filesystem.Path;
|
||||
|
||||
namespace Compression.BSA
|
||||
@ -141,6 +142,20 @@ namespace Compression.BSA
|
||||
return ((ulong) (hash2 + hash3) << 32) + hash1;
|
||||
}
|
||||
|
||||
public static async Task CopyToLimitAsync(this Stream frm, Stream tw, int limit)
|
||||
{
|
||||
var buff = new byte[1024];
|
||||
while (limit > 0)
|
||||
{
|
||||
var to_read = Math.Min(buff.Length, limit);
|
||||
var read = await frm.ReadAsync(buff, 0, to_read);
|
||||
await tw.WriteAsync(buff, 0, read);
|
||||
limit -= read;
|
||||
}
|
||||
|
||||
tw.Flush();
|
||||
}
|
||||
|
||||
public static void CopyToLimit(this Stream frm, Stream tw, int limit)
|
||||
{
|
||||
var buff = new byte[1024];
|
||||
|
||||
62
README.md
62
README.md
@ -2,18 +2,18 @@
|
||||
|
||||
[](https://dev.azure.com/tbaldridge/tbaldridge/_build/latest?definitionId=3&branchName=master)
|
||||
|
||||
The general idea behind this program is fairly simple. Given a Mod Organizer 2 folder and profile, generate list of instructions that will allow
|
||||
The general idea behind this program is fairly simple. Given a Mod Organizer 2 folder and profile, generate a list of instructions that will allow
|
||||
a program to automatically recreate the contents of the folder on another machine. Think of it as replication, but without ever distributing copyrighted
|
||||
files or syncing data between the source and destination machine. The end result is a program that recreate a modlist on a computer while respecting the
|
||||
files or syncing data between the source and destination machine. The end result is a program that recreates a modlist on a computer while respecting the
|
||||
rights of the game publisher and the mod authors.
|
||||
|
||||
### Installing a Modlist
|
||||
### Installing a modlist
|
||||
Please visit our Discord link below for information on how to obtain and install a modlist.
|
||||
|
||||
### Social Links
|
||||
|
||||
- [Discord](https://discord.gg/zgbrkmA)
|
||||
- [Patreon](https://www.patreon.com/user?u=11907933) Check this page for updates and to vote on features
|
||||
- [Discord](https://discord.gg/zgbrkmA) Obtain modlists here, ask for support or have a friendly chat with fellow modders.
|
||||
- [Patreon](https://www.patreon.com/user?u=11907933) Check out this page for update posts, patrons are able to vote on the direction of development.
|
||||
|
||||
### What Wabbajack can do
|
||||
|
||||
@ -26,13 +26,13 @@ let's do a rundown of all the supported features:
|
||||
- Skyrim SE
|
||||
- Skyrim LE
|
||||
- Support for automatic downloads from the following sources
|
||||
- Nexus Mods (Premium accounts only)
|
||||
- Nexus Mods (Premium accounts only, due to specific API calls)
|
||||
- Dropbox
|
||||
- Google Drive
|
||||
- Mega
|
||||
- ModDB
|
||||
- Direct URLs (with custom header support)
|
||||
- Support the following archive types
|
||||
- Supports the following archive types
|
||||
- `.zip`
|
||||
- `.7z`
|
||||
- `.rar`
|
||||
@ -44,9 +44,9 @@ let's do a rundown of all the supported features:
|
||||
- Renamed/deleted/moved files are detected and handled
|
||||
- Multiple mods installed into the same mod folder
|
||||
- A mod split across multiple mod folders
|
||||
- Any tools installed in the MO2 folder. Want your users to have BethIni or xEdit? Just put them in a folder inside the MO2 install folder
|
||||
- ENBseries files that exist in the game folder
|
||||
- SKSE install
|
||||
- ENBSeries files that exist in the game folder
|
||||
- Any tools installed in the MO2 folder. Want your users to have BethINI or xEdit? Just put them in a folder inside the MO2 install folder
|
||||
- SKSE
|
||||
- The following situations are automatically detected and handled by the automated binary patcher (not an exhaustive list)
|
||||
- ESP cleaning
|
||||
- form 44 conversion
|
||||
@ -75,7 +75,7 @@ authors to create a permissions system Wabbajack will follow during installation
|
||||
can be found on [Github](https://github.com/wabbajack-tools/opt-out-lists/blob/master/NexusModPermissions.yml). Feel free to contact
|
||||
us via discord about any questions you may have.
|
||||
|
||||
### Creating a ModList Installer
|
||||
### Creating A Modlist Installer
|
||||
|
||||
Overview video [`https://www.youtube.com/watch?v=5Fwr0Chtcuc`](https://www.youtube.com/watch?v=5Fwr0Chtcuc)
|
||||
|
||||
@ -92,21 +92,23 @@ continuum).
|
||||
3) Now load Wabbajack, and point it to the `\<MO2 Folder>\mods\<your profile>\modlist.txt` file.
|
||||
4) Click `Begin`.
|
||||
5) Wabbajack will start by indexing all your downloaded archives. This will take some time on most machines as the application
|
||||
has to performa `SHA-256` hash on every file in every archive. However the results of this operation are cached, so you'll only need
|
||||
has to perform a `SHA-256` hash on every file in every archive. However the results of this operation are cached, so you'll only need
|
||||
to do this once for every downloaded file.
|
||||
6) Once completed, Wabbajack will collect the files required for the modlist install and begin running them through the compilation stack.
|
||||
7) If all goes well, you should see a new `<your profile name>.exe` file next to `Wabbajack.exe` that you just ran. This new `.exe` is the one
|
||||
you want to hand out as a auto modlist installer.
|
||||
7) If all goes well, you should see a new `<your profile name>.wabbajack` file next to `Wabbajack.exe` that you just ran. This new `.wabbajack` file is the one
|
||||
you want to hand out, users can select the file in Wabbajack and install your modlist.
|
||||
|
||||
### Installing a ModList
|
||||
### Installing A Modlist
|
||||
|
||||
1) Get a modlist installer, it's a `.exe` file that was created by Wabbajack
|
||||
2) Run the `.exe`, the install folder defaults to the same folder as the executable, change it if you want.
|
||||
3) Click `Begin` to start installation. At some point you will be prompted for SSO authorization on the Nexus, files
|
||||
will be auto installed and downloaded
|
||||
4) After installation has completed, run `Mod Organizer 2.exe`, select `Portable` and your game type.
|
||||
There are two options for installing a modlist. You can either
|
||||
1a) Install a modlist from the built-in menu. Simply select your preferred option in Wabbajack and click on 'Download and Install'.
|
||||
1b) Get a Wabbajack Modlist installer file (ending in `.wabbajack`), and install it by clicking the 'Install from Disk' button in Wabbajack.
|
||||
2) Click `Begin` to start installation. At some point you will be prompted for SSO authorization on the Nexus, after authorization all of the files
|
||||
will be automatically downloaded and installed.
|
||||
3) After installation has completed, run `Mod Organizer 2.exe` in the installation folder, and make sure to select `Portable` and your game type.
|
||||
If Mod Organizer does not show you these options when starting it, click on the top left button and you should see the instance menu.
|
||||
|
||||
### How it works
|
||||
### How It Works
|
||||
|
||||
At a technical level the process is as follows.
|
||||
|
||||
@ -150,27 +152,27 @@ Currently the Resolution stack looks like this:
|
||||
27) Ignore `splash.png` it's created for some games (like FO4) by MO2
|
||||
28) Error for any file that survives to this point.
|
||||
|
||||
So as you can see we handle a lot of possible install situations. See the section on [`Creating a Modpack`](README.md#Creating_a_ModList_Installer) for information on working with the installer
|
||||
So as you can see we handle a lot of possible installation situations. See the section on [`Creating a Modlist`](README.md#Creating_a_modlist_installer) for information on creating an installer.
|
||||
|
||||
### Wabbajack Flags
|
||||
|
||||
The if the following words are found in a mod's notes or comments they trigger special behavior in Wabbajack.
|
||||
|
||||
- `WABBAJACK_INCLUDE` - All the files int he mod will be inlined into the installer
|
||||
- `WABBAJAC_ALWAYS_ENABLE` - The mod's files will be considered by the compiler even if the mod is disabled in the profile
|
||||
- `WABBAJACK_INCLUDE` - All the files in the mod will be inlined into the installer
|
||||
- `WABBAJACK_ALWAYS_ENABLE` - The mod's files will be considered by the compiler even if the mod is disabled in the profile
|
||||
|
||||
### Patches
|
||||
|
||||
Wabbajack can create binary patches for files that have been modified after installation. This could be `.esp` files that have been cleaned or patched. Or
|
||||
it could be meshes and textures that have been optimized to work better in a given game. In any case a BSDiff file is generated. The output of this process
|
||||
is copied directly into the modlist instructions. However! It is important to note that the patch file is 100% useless without the source file. So `original + patch = final_file`. Without the original file, the final file cannot be recrated. This allows us to distribute arbitrary changes without violating copyrights as we do not copy
|
||||
copyrighted material. Instead we copy instructions on how to modify the copyrighted material.
|
||||
is copied directly into the modlist instructions. However! It is important to note that the patch file is 100% useless without the source file. So `original + patch = final_file`. Without the original file, the final file cannot be recreated. This allows us to distribute arbitrary changes without violating copyrights as we do not copy
|
||||
copyrighted material. Instead, we copy instructions on how to modify the copyrighted material.
|
||||
|
||||
### FAQ
|
||||
|
||||
**How do I get Wabbjack to handle mods from `X`**
|
||||
**How do I get Wabbajack to handle mods from `X`?
|
||||
|
||||
Look at the [`RECIPES.md`] file, we keep a knowledgebase of how to deal with given types of mods in that file.
|
||||
Look at the [`RECIPES.md`] file, we keep a knowledge base of how to deal with given types of mods in that file.
|
||||
|
||||
**How do I contribute to Wabbajack?**
|
||||
|
||||
@ -182,7 +184,7 @@ Self-contained folders are a cleaner abstraction than dumping tons of modlists i
|
||||
and MO2 really isn't designed to support lots of disparate modlists. For example if two modlists both wanted a given texture mod, but different options they would
|
||||
somehow have to keep the names of their mods separate. MO2 isn't that big of an app, so there's really no reason not to install a new copy for each modlist.
|
||||
|
||||
**Why don't I see any mods when I open Mod Organizer 2 after install?**
|
||||
**Why don't I see any mods when I open Mod Organizer 2 after the modlist installation has finished?**
|
||||
|
||||
Make sure you selected the "Portable" mode when starting MO2 for the first time. In addition, make sure you haven't installed MO2 in a non-portable way on the same box.
|
||||
Really, always use "Portable Mode" it's cleaner and there really isn't a reason not too do so. Make the data self-contained. It's cleaner that way.
|
||||
@ -202,7 +204,7 @@ The end result is an app with a ton of features, and a less than professional UI
|
||||
|
||||
## Thanks to
|
||||
|
||||
Our tester and Discord members who encourage development and help test the builds.
|
||||
Our testers and Discord members who encourage development and help test the builds.
|
||||
|
||||
### Patreon Supporters
|
||||
|
||||
|
||||
77
RECIPES.md
77
RECIPES.md
@ -1,33 +1,33 @@
|
||||
## Instructions on how to configure specific mods/utilities for use in Wabbajack
|
||||
|
||||
#### Manually downloaded Nexus files
|
||||
MO2 can get archive info from random files in the download folder
|
||||
MO2 can retrieve archive information from files in the downloads folder.
|
||||
|
||||
1) Download the file manually
|
||||
2) Copy the file into the MO2 downloads folder
|
||||
3) In MO2 click `Query Info` on the file (may need to refresh the GUI)
|
||||
1) Download the file manually.
|
||||
2) Copy the file into the MO2 downloads folder.
|
||||
3) In MO2 click `Query Info` on the file (you may need to refresh the GUI by hitting F5).
|
||||
4) The red icon should go away if MO2 found the query info.
|
||||
5) Wabbajack will find the info queried by MO2
|
||||
5) Wabbajack will find the mod information queried by MO2.
|
||||
|
||||
#### Mod Organizer 2
|
||||
Comes from the Nexus as a Nexus mod.
|
||||
|
||||
1) Download MO2 (archive version)
|
||||
2) Extract it into a folder
|
||||
3) Start it in portable version and select your game
|
||||
4) Copy the MO2 archive into the `downloads` folder in the MO2 install folder
|
||||
5) Open MO2 go to downloads, right click on the MO2 archive and click `Query Info` MO2 will
|
||||
automatically link the file to the nexus modID and fileID. These values will be found and used
|
||||
by wabbjack
|
||||
1) Download MO2 (archive version).
|
||||
2) Extract it into a folder.
|
||||
3) Start it in portable version and select your game.
|
||||
4) Copy the MO2 archive into the `downloads` folder in the MO2 install folder.
|
||||
5) Open MO2 go to downloads, right click on the MO2 archive and click `Query Info`. MO2 will
|
||||
automatically link the file to the Nexus Mod ID and File ID. These values will be found and used
|
||||
by Wabbajack.
|
||||
|
||||
#### SKSE
|
||||
SKSE Servers support direct URL downloads
|
||||
SKSE servers support direct URL downloads.
|
||||
|
||||
1) Download SKSE and copy the binary files into your game folder
|
||||
1) Download SKSE and copy the binary files into your game folder.
|
||||
2) Copy the SKSE archive into the `downloads` folder in MO2.
|
||||
3) Install the SKSE archive through MO2 from the downloads tab
|
||||
3) Install the SKSE archive through MO2 from the downloads tab.
|
||||
4) In a file browser, go to the MO2 `downloads` folder and load the `skse*.7z.meta` file, and add
|
||||
the following line to the `[General]` section, changing the URL to the exact URL you used to download skse:
|
||||
the following line to the `[General]` section, changing the URL to the exact URL you used to download SKSE:
|
||||
|
||||
```ini
|
||||
directURL=https://skse.silverlock.org/beta/skse64_2_00_16.7z
|
||||
@ -42,47 +42,44 @@ uninstalled=false
|
||||
directURL=https://skse.silverlock.org/beta/skse64_2_00_16.7z
|
||||
```
|
||||
|
||||
#### ENB Series
|
||||
ENB Series Servers support direct downloads as long as a header is provided.
|
||||
#### ENBSeries
|
||||
ENBSeries servers support direct downloads as long as a header is provided.
|
||||
|
||||
1) Download the ENBSeries archive and copy the binary files into your game folder
|
||||
2) Copy the ENBSeries archive into your downloads folder in MO2
|
||||
1) Download the ENBSeries archive and copy the binary files into your game folder.
|
||||
2) Copy the ENBSeries archive into your downloads folder in MO2.
|
||||
3) Create a `.meta` file with the same name prefix as the ENBSeries archive name. So if the ENBSeries
|
||||
archive is named `enb-foo.bar.7z` make a text file named `enb-foo.bar.7z.meta`
|
||||
archive is named `enb-foo.bar.7z` make a text file named `enb-foo.bar.7z.meta`.
|
||||
4) Update the contents of the `.meta` file to look like this, updating the URL to the exact URL
|
||||
used to download the ENB (not the ENB landing page, but the download URL)
|
||||
5) The Referer value may need to be updated for games that are not Skyrim SE
|
||||
used to download the ENB (not the ENB landing page, but the download URL).
|
||||
5) The Referer value may need to be updated for games that are not Skyrim SE.
|
||||
|
||||
```ini
|
||||
[General]
|
||||
installed=true
|
||||
uninstalled=false
|
||||
directURL=http://enbdev.com/enbseries_skyrimse_v0390.zip
|
||||
directURL=http://enbdev.com/enbseries_skyrimse_v0394.zip
|
||||
directURLHeaders=Referer:http://enbdev.com/download_mod_tesskyrimse.html
|
||||
```
|
||||
|
||||
**NOTE:** The author of ENBSeries has a habit of updating the program without changing the version number. If you get
|
||||
install errors stating that the hash of the downloaded file wasn't what was expected, update your ENBSeries to the latest
|
||||
version and recompile the modlist.
|
||||
installation errors stating that the hash of the downloaded file wasn't what was expected, redownload ENBSeries to update to the new version and recompile the modlist.
|
||||
|
||||
#### SSE Engine Fixes
|
||||
Both files come from the nexus.
|
||||
Both files come from the Nexus.
|
||||
|
||||
1) Download and install Part 1 via MO2
|
||||
2) Download Part 2 manually, copy the binaries into the MO2 folder
|
||||
3) Copy the Part 2 archive into the `downloads` folder in MO2
|
||||
4) Use MO2 to `Query Info` on the Part 2 file
|
||||
|
||||
#### SSEdit BethINI
|
||||
1) Download and install Part 1 via MO2.
|
||||
2) Download Part 2 manually, copy the binaries into the MO2 folder.
|
||||
3) Copy the Part 2 archive into the `downloads` folder in MO2.
|
||||
4) Use MO2 to `Query Info` on the Part 2 file.
|
||||
|
||||
#### Skyrim Realistic Overhaul
|
||||
SRO is only stored on the ModDB servers
|
||||
SRO is only stored on the ModDB servers.
|
||||
|
||||
1) Download the files manually
|
||||
2) Copy them into the MO2 downloads folder
|
||||
3) Install them via MO2
|
||||
4) For each file, go back to ModDB, and right click on the download button clicking "copy link as"
|
||||
5) Update the `.meta` file for each archive to look something like this
|
||||
1) Download the files manually.
|
||||
2) Copy them into the MO2 downloads folder.
|
||||
3) Install them via MO2.
|
||||
4) For each file, go back to ModDB, and right click on the download button clicking "Copy Link As"
|
||||
5) Update the `.meta` file for each archive to look something like this.
|
||||
|
||||
```ini
|
||||
[General]
|
||||
@ -100,4 +97,4 @@ If you have a file that comes from Dropbox, the process is simple:
|
||||
1) Download the file
|
||||
2) Copy it into the MO2 downloads folder
|
||||
3) Install it via MO2
|
||||
4) Update the .meta file for the archive to point to the dropbox URL
|
||||
4) Update the .meta file for the archive to point to the Dropbox URL
|
||||
|
||||
@ -368,7 +368,7 @@ namespace VFS
|
||||
var tmp_dir = Path.Combine(_stagedRoot, Guid.NewGuid().ToString());
|
||||
Utils.Status($"Extracting Archive {Path.GetFileName(f.StagedPath)}");
|
||||
|
||||
FileExtractor.ExtractAll(f.StagedPath, tmp_dir);
|
||||
FileExtractor.ExtractAll(f.StagedPath, tmp_dir).Wait();
|
||||
|
||||
|
||||
Utils.Status($"Updating Archive {Path.GetFileName(f.StagedPath)}");
|
||||
@ -422,7 +422,7 @@ namespace VFS
|
||||
foreach (var group in grouped)
|
||||
{
|
||||
var tmp_path = Path.Combine(_stagedRoot, Guid.NewGuid().ToString());
|
||||
FileExtractor.ExtractAll(group.Key.StagedPath, tmp_path);
|
||||
FileExtractor.ExtractAll(group.Key.StagedPath, tmp_path).Wait();
|
||||
Paths.Add(tmp_path);
|
||||
foreach (var file in group)
|
||||
file._stagedPath = Path.Combine(tmp_path, file.Paths[group.Key.Paths.Length]);
|
||||
|
||||
86
Wabbajack.Common.CSP/AChannel.cs
Normal file
86
Wabbajack.Common.CSP/AChannel.cs
Normal file
@ -0,0 +1,86 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public abstract class AChannel<TIn, TOut> : IChannel<TIn, TOut>
|
||||
{
|
||||
public abstract bool IsClosed { get; }
|
||||
public abstract void Close();
|
||||
public abstract (AsyncResult, bool) Put(TIn val, Handler<Action<bool>> handler);
|
||||
public abstract (AsyncResult, TOut) Take(Handler<Action<bool, TOut>> handler);
|
||||
|
||||
private Task<(bool, TOut)> _take_cancelled_task;
|
||||
|
||||
private Task<(bool, TOut)> TakeCancelledTask
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_take_cancelled_task == null)
|
||||
_take_cancelled_task = Task.FromCanceled<(bool, TOut)>(CancellationToken.None);
|
||||
return _take_cancelled_task;
|
||||
}
|
||||
}
|
||||
|
||||
private Task<bool> _put_cancelled_task;
|
||||
|
||||
private Task<bool> PutCancelledTask
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_put_cancelled_task == null)
|
||||
_put_cancelled_task = Task.FromCanceled<bool>(CancellationToken.None);
|
||||
return _put_cancelled_task;
|
||||
}
|
||||
}
|
||||
|
||||
public ValueTask<(bool, TOut)> Take(bool onCaller)
|
||||
{
|
||||
var handler = new TakeTaskHandler<TOut>();
|
||||
var (resultType, val) = Take(handler);
|
||||
|
||||
switch (resultType)
|
||||
{
|
||||
case AsyncResult.Closed:
|
||||
return new ValueTask<(bool, TOut)>((false, default));
|
||||
case AsyncResult.Completed:
|
||||
return new ValueTask<(bool, TOut)>((true, val));
|
||||
case AsyncResult.Enqueued:
|
||||
return new ValueTask<(bool, TOut)>(handler.TaskCompletionSource.Task);
|
||||
case AsyncResult.Canceled:
|
||||
return new ValueTask<(bool, TOut)>(TakeCancelledTask);
|
||||
default:
|
||||
// Should never happen
|
||||
throw new InvalidDataException();
|
||||
}
|
||||
}
|
||||
|
||||
public ValueTask<bool> Put(TIn val, bool onCaller)
|
||||
{
|
||||
var handler = new PutTaskHandler<bool>();
|
||||
var (resultType, putResult) = Put(val, handler);
|
||||
|
||||
switch (resultType)
|
||||
{
|
||||
case AsyncResult.Completed:
|
||||
return new ValueTask<bool>(putResult);
|
||||
case AsyncResult.Canceled:
|
||||
return new ValueTask<bool>(PutCancelledTask);
|
||||
case AsyncResult.Closed:
|
||||
return new ValueTask<bool>(false);
|
||||
case AsyncResult.Enqueued:
|
||||
return new ValueTask<bool>(handler.TaskCompletionSource.Task);
|
||||
default:
|
||||
// Should never happen
|
||||
throw new InvalidDataException();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
31
Wabbajack.Common.CSP/AsyncResult.cs
Normal file
31
Wabbajack.Common.CSP/AsyncResult.cs
Normal file
@ -0,0 +1,31 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public enum AsyncResult : int
|
||||
{
|
||||
/// <summary>
|
||||
/// The channel was closed, so the returned value is meaningless
|
||||
/// </summary>
|
||||
Closed,
|
||||
|
||||
/// <summary>
|
||||
/// The handler was canceled, so the returned value is meaningless
|
||||
/// </summary>
|
||||
Canceled,
|
||||
|
||||
/// <summary>
|
||||
/// The callback was enqueued into the pending operations buffer, return value is useless
|
||||
/// </summary>
|
||||
Enqueued,
|
||||
|
||||
/// <summary>
|
||||
/// The operation passed on the current thread, return the current value as the response value
|
||||
/// </summary>
|
||||
Completed
|
||||
}
|
||||
}
|
||||
42
Wabbajack.Common.CSP/CSP Readme.md
Normal file
42
Wabbajack.Common.CSP/CSP Readme.md
Normal file
@ -0,0 +1,42 @@
|
||||
### Overview of CSP (Communicating Sequential Processes) for the C# programmer
|
||||
|
||||
#### What is CSP?
|
||||
Communicating Sequential processes is a programming model invented in 1978 by Tony Hoare, who described a process
|
||||
of computation where hundreds or thousands of small processes communicate via channels. Think of this process like
|
||||
a assembly line. Each worker in the factory is a process, and the conveyor belts are the channels. The workers don't need
|
||||
to know where a part came from, or where it's going, they simply take one item off the belt, perform an operation and pass
|
||||
the item on to another belt. This analogy works quite well, and the following observations about a factory also apply to
|
||||
CSP:
|
||||
|
||||
* Multiple workers can pull from the same belt (channel/queue)
|
||||
* Multiple workers can put work onto the belt
|
||||
* Belts can buffer items, for slow consumers, but at some point they backup and block the writer
|
||||
* A worker can pull/push to multiple belts.
|
||||
|
||||
|
||||
#### What does this look like in C#?
|
||||
|
||||
The basic unit of CSP in this library is the channel:
|
||||
|
||||
```
|
||||
var chan = Channel.Create()
|
||||
```
|
||||
|
||||
Without any other parameters this creates a channel with a size of 0, so every pending put must be matched
|
||||
1:1 with a take. This creates a syncronization point. Channels are fully async and thread-safe:
|
||||
|
||||
```
|
||||
public async Task TestTakePutBlocking()
|
||||
{
|
||||
var channel = Channel.Create<int>();
|
||||
// Channel size is 0, so we can't await, because we'd never complete
|
||||
var ptask = channel.Put(1);
|
||||
|
||||
// Now the put is dispatched to the scheduler because we've taken the value
|
||||
var (open, val) = await channel.Take();
|
||||
|
||||
Assert.AreEqual(1, val);
|
||||
Assert.IsTrue(open);
|
||||
Assert.IsTrue(await ptask);
|
||||
}
|
||||
```
|
||||
76
Wabbajack.Common.CSP/Channel.cs
Normal file
76
Wabbajack.Common.CSP/Channel.cs
Normal file
@ -0,0 +1,76 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Data;
|
||||
using System.Linq;
|
||||
using System.Reactive.Linq;
|
||||
using System.Reactive.Subjects;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public static class Channel
|
||||
{
|
||||
/// <summary>
|
||||
/// Creates a channel without a buffer, and with no conversion function. This provides a syncronization
|
||||
/// point, where all puts are matched 1:1 with takes.
|
||||
/// </summary>
|
||||
/// <param name="bufferSize"></param>
|
||||
/// <typeparam name="T">The type of values transferred by the channel</typeparam>
|
||||
/// <returns>A new channel</returns>
|
||||
public static IChannel<T, T> Create<T>()
|
||||
{
|
||||
return new ManyToManyChannel<T, T>(x => x);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a channel with a given enumerator as the starting buffer. Values will not be puttable into this channel
|
||||
/// and it will start closed. This is a easy way to spool a collection onto a channel. Note: the enumerator will be
|
||||
/// run inside the channel's lock, so it may not be wise to pass in an enumerator that performs heavy computation.
|
||||
/// </summary>
|
||||
/// <param name="e">A IEnumerator to use as the contents of the channel</param>
|
||||
/// <typeparam name="T">The type of values transferred by the channel</typeparam>
|
||||
/// <returns>A new channel</returns>
|
||||
public static IChannel<T, T> Create<T>(IEnumerator<T> e)
|
||||
{
|
||||
var chan = new ManyToManyChannel<T, T>(x => x, (_, __) => false, _ => {}, new EnumeratorBuffer<T>(e));
|
||||
chan.Close();
|
||||
return chan;
|
||||
}
|
||||
|
||||
|
||||
public static IChannel<T, T> Create<T>(int buffer_size)
|
||||
{
|
||||
var buffer = new FixedSizeBuffer<T>(buffer_size);
|
||||
return new ManyToManyChannel<T, T>(x => x, (buff, itm) =>
|
||||
{
|
||||
buff.Add(itm);
|
||||
return false;
|
||||
},
|
||||
b => {}, buffer);
|
||||
}
|
||||
|
||||
public static IChannel<TIn, TOut> Create<TIn, TOut>(int buffer_size, Func<IObservable<TIn>, IObservable<TOut>> transform)
|
||||
{
|
||||
var buf = new RxBuffer<TIn, TOut>(buffer_size, transform);
|
||||
return ChannelForRxBuf(buf);
|
||||
}
|
||||
|
||||
private static ManyToManyChannel<TIn, TOut> ChannelForRxBuf<TIn, TOut>(RxBuffer<TIn, TOut> buf)
|
||||
{
|
||||
return new ManyToManyChannel<TIn, TOut>(null, RxBuffer<TIn,TOut>.TransformAdd, RxBuffer<TIn, TOut>.Finalize, buf);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Creates a channel that discards every value
|
||||
/// </summary>
|
||||
/// <typeparam name="TIn"></typeparam>
|
||||
/// <returns></returns>
|
||||
public static IChannel<TIn, TIn> CreateSink<TIn>()
|
||||
{
|
||||
var buf = new RxBuffer<TIn, TIn>(1, e => e.Where(itm => false));
|
||||
return ChannelForRxBuf(buf);
|
||||
}
|
||||
}
|
||||
}
|
||||
39
Wabbajack.Common.CSP/EnumeratorBuffer.cs
Normal file
39
Wabbajack.Common.CSP/EnumeratorBuffer.cs
Normal file
@ -0,0 +1,39 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
class EnumeratorBuffer<T> : IBuffer<T>
|
||||
{
|
||||
private readonly IEnumerator<T> _enumerator;
|
||||
private bool _empty;
|
||||
|
||||
public EnumeratorBuffer(IEnumerator<T> enumerator)
|
||||
{
|
||||
_enumerator = enumerator;
|
||||
_empty = !_enumerator.MoveNext();
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
|
||||
public bool IsFull => true;
|
||||
public bool IsEmpty => _empty;
|
||||
public T Remove()
|
||||
{
|
||||
var val = _enumerator.Current;
|
||||
_empty = !_enumerator.MoveNext();
|
||||
return val;
|
||||
}
|
||||
|
||||
public void Add(T itm)
|
||||
{
|
||||
throw new InvalidDataException();
|
||||
}
|
||||
}
|
||||
}
|
||||
168
Wabbajack.Common.CSP/Extensions.cs
Normal file
168
Wabbajack.Common.CSP/Extensions.cs
Normal file
@ -0,0 +1,168 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public static class CSPExtensions
|
||||
{
|
||||
public static async Task OntoChannel<T>(this IEnumerable<T> coll, IWritePort<T> chan)
|
||||
{
|
||||
foreach (var val in coll)
|
||||
{
|
||||
if (!await chan.Put(val)) break;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Turns a IEnumerable collection into a channel. Note, computation of the enumerable will happen inside
|
||||
/// the lock of the channel, so try to keep the work of the enumerable light.
|
||||
/// </summary>
|
||||
/// <typeparam name="T"></typeparam>
|
||||
/// <param name="coll">Collection to spool out of the channel.</param>
|
||||
/// <returns></returns>
|
||||
public static IReadPort<T> ToChannel<T>(this IEnumerable<T> coll)
|
||||
{
|
||||
var chan = Channel.Create(coll.GetEnumerator());
|
||||
chan.Close();
|
||||
return chan;
|
||||
}
|
||||
|
||||
public static IReadPort<TOut> Select<TIn, TOut>(this IReadPort<TIn> from, Func<TIn, Task<TOut>> f, bool propagateClose = true)
|
||||
{
|
||||
var to = Channel.Create<TOut>(4);
|
||||
Task.Run(async () =>
|
||||
{
|
||||
try
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var (is_open_src, val) = await from.Take();
|
||||
if (!is_open_src) break;
|
||||
|
||||
var is_open_dest = await to.Put(await f(val));
|
||||
if (!is_open_dest) break;
|
||||
}
|
||||
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (propagateClose)
|
||||
{
|
||||
from.Close();
|
||||
to.Close();
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
return to;
|
||||
}
|
||||
|
||||
public static async Task UnorderedParallelDo<T>(this IEnumerable<T> coll, Func<T, Task> f)
|
||||
{
|
||||
var sink = Channel.CreateSink<bool>();
|
||||
await coll.ToChannel()
|
||||
.UnorderedPipeline(Environment.ProcessorCount,
|
||||
sink,
|
||||
async itm =>
|
||||
{
|
||||
await f(itm);
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Takes all the values from chan, once the channel closes returns a List of the values taken.
|
||||
/// </summary>
|
||||
/// <typeparam name="TOut"></typeparam>
|
||||
/// <typeparam name="TIn"></typeparam>
|
||||
/// <param name="chan"></param>
|
||||
/// <returns></returns>
|
||||
public static async Task<List<T>> TakeAll<T>(this IReadPort<T> chan)
|
||||
{
|
||||
List<T> acc = new List<T>();
|
||||
while (true)
|
||||
{
|
||||
var (open, val) = await chan.Take();
|
||||
|
||||
if (!open) break;
|
||||
|
||||
acc.Add(val);
|
||||
}
|
||||
return acc;
|
||||
}
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Pipes values from `from` into `to`
|
||||
/// </summary>
|
||||
/// <typeparam name="TIn"></typeparam>
|
||||
/// <typeparam name="TMid"></typeparam>
|
||||
/// <typeparam name="TOut"></typeparam>
|
||||
/// <param name="from">source channel</param>
|
||||
/// <param name="to">destination channel</param>
|
||||
/// <param name="closeOnFinished">Tf true, will close the other channel when one channel closes</param>
|
||||
/// <returns></returns>
|
||||
public static async Task Pipe<T>(this IReadPort<T> from, IWritePort<T> to, bool closeOnFinished = true)
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var (isFromOpen, val) = await from.Take();
|
||||
if (isFromOpen)
|
||||
{
|
||||
var isToOpen = await to.Put(val);
|
||||
if (isToOpen) continue;
|
||||
if (closeOnFinished)
|
||||
@from.Close();
|
||||
break;
|
||||
}
|
||||
if (closeOnFinished)
|
||||
to.Close();
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
public static Task<T> ThreadedTask<T>(Func<T> action)
|
||||
{
|
||||
var src = new TaskCompletionSource<T>();
|
||||
var th = new Thread(() =>
|
||||
{
|
||||
try
|
||||
{
|
||||
src.SetResult(action());
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
src.SetException(ex);
|
||||
}
|
||||
}) {Priority = ThreadPriority.BelowNormal};
|
||||
th.Start();
|
||||
return src.Task;
|
||||
}
|
||||
|
||||
public static Task ThreadedTask<T>(Action action)
|
||||
{
|
||||
var src = new TaskCompletionSource<bool>();
|
||||
var th = new Thread(() =>
|
||||
{
|
||||
try
|
||||
{
|
||||
action();
|
||||
src.SetResult(true);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
src.SetException(ex);
|
||||
}
|
||||
})
|
||||
{ Priority = ThreadPriority.BelowNormal };
|
||||
th.Start();
|
||||
return src.Task;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
36
Wabbajack.Common.CSP/FixedSizeBuffer.cs
Normal file
36
Wabbajack.Common.CSP/FixedSizeBuffer.cs
Normal file
@ -0,0 +1,36 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public class FixedSizeBuffer<T> : IBuffer<T>
|
||||
{
|
||||
private int _size;
|
||||
private RingBuffer<T> _buffer;
|
||||
|
||||
public FixedSizeBuffer(int size)
|
||||
{
|
||||
_size = size;
|
||||
_buffer = new RingBuffer<T>(size);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
}
|
||||
|
||||
public bool IsFull => _buffer.Length >= _size;
|
||||
public bool IsEmpty => _buffer.IsEmpty;
|
||||
public T Remove()
|
||||
{
|
||||
return _buffer.Pop();
|
||||
}
|
||||
|
||||
public void Add(T itm)
|
||||
{
|
||||
_buffer.UnboundedUnshift(itm);
|
||||
}
|
||||
}
|
||||
}
|
||||
34
Wabbajack.Common.CSP/Handler.cs
Normal file
34
Wabbajack.Common.CSP/Handler.cs
Normal file
@ -0,0 +1,34 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using System.Threading.Tasks.Sources;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface Handler<T>
|
||||
{
|
||||
/// <summary>
|
||||
/// Returns true if this handler has a callback, must work without a lock
|
||||
/// </summary>
|
||||
bool IsActive { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Returns true if this handler may be blocked, otherwise it must not block
|
||||
/// </summary>
|
||||
bool IsBlockable { get; }
|
||||
|
||||
/// <summary>
|
||||
/// A unique id for lock aquisition order, 0 if no lock
|
||||
/// </summary>
|
||||
uint LockId { get; }
|
||||
|
||||
/// <summary>
|
||||
/// Commit to fulfilling its end of the transfer, returns cb, must be called within a lock
|
||||
/// </summary>
|
||||
/// <returns>A callback</returns>
|
||||
T Commit();
|
||||
}
|
||||
|
||||
}
|
||||
12
Wabbajack.Common.CSP/IBuffer.cs
Normal file
12
Wabbajack.Common.CSP/IBuffer.cs
Normal file
@ -0,0 +1,12 @@
|
||||
using System;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface IBuffer<T> : IDisposable
|
||||
{
|
||||
bool IsFull { get; }
|
||||
bool IsEmpty { get; }
|
||||
T Remove();
|
||||
void Add(T itm);
|
||||
}
|
||||
}
|
||||
12
Wabbajack.Common.CSP/IChannel.cs
Normal file
12
Wabbajack.Common.CSP/IChannel.cs
Normal file
@ -0,0 +1,12 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface IChannel<TIn, TOut> : IReadPort<TOut>, IWritePort<TIn>
|
||||
{
|
||||
}
|
||||
}
|
||||
14
Wabbajack.Common.CSP/ICloseable.cs
Normal file
14
Wabbajack.Common.CSP/ICloseable.cs
Normal file
@ -0,0 +1,14 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface ICloseable
|
||||
{
|
||||
bool IsClosed { get; }
|
||||
void Close();
|
||||
}
|
||||
}
|
||||
15
Wabbajack.Common.CSP/IReadPort.cs
Normal file
15
Wabbajack.Common.CSP/IReadPort.cs
Normal file
@ -0,0 +1,15 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface IReadPort<TOut> : ICloseable
|
||||
{
|
||||
ValueTask<(bool, TOut)> Take(bool onCaller = true);
|
||||
(AsyncResult, TOut) Take(Handler<Action<bool, TOut>> handler);
|
||||
|
||||
}
|
||||
}
|
||||
14
Wabbajack.Common.CSP/IWritePort.cs
Normal file
14
Wabbajack.Common.CSP/IWritePort.cs
Normal file
@ -0,0 +1,14 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public interface IWritePort<TIn> : ICloseable
|
||||
{
|
||||
(AsyncResult, bool) Put(TIn val, Handler<Action<bool>> handler);
|
||||
ValueTask<bool> Put(TIn val, bool onCaller = true);
|
||||
}
|
||||
}
|
||||
390
Wabbajack.Common.CSP/ManyToManyChannel.cs
Normal file
390
Wabbajack.Common.CSP/ManyToManyChannel.cs
Normal file
@ -0,0 +1,390 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
/// <summary>
|
||||
/// An almost 1:1 port of Clojure's core.async channels
|
||||
/// </summary>
|
||||
|
||||
public class ManyToManyChannel<TIn, TOut> : AChannel<TIn, TOut>
|
||||
{
|
||||
public const int MAX_QUEUE_SIZE = 1024;
|
||||
|
||||
private RingBuffer<Handler<Action<bool, TOut>>> _takes = new RingBuffer<Handler<Action<bool, TOut>>>(8);
|
||||
private RingBuffer<(Handler<Action<bool>>, TIn)> _puts = new RingBuffer<(Handler<Action<bool>>, TIn)>(8);
|
||||
private IBuffer<TOut> _buf;
|
||||
private Func<IBuffer<TOut>, TIn, bool> _add;
|
||||
private Action<IBuffer<TOut>> _finalize;
|
||||
private Func<TIn, TOut> _converter;
|
||||
volatile bool _isClosed = false;
|
||||
|
||||
public ManyToManyChannel(Func<TIn, TOut> converter)
|
||||
{
|
||||
_buf = null;
|
||||
_add = null;
|
||||
_finalize = null;
|
||||
_converter = converter;
|
||||
}
|
||||
|
||||
public ManyToManyChannel(Func<TIn, TOut> converter, Func<IBuffer<TOut>, TIn, bool> add, Action<IBuffer<TOut>> finalize, IBuffer<TOut> buffer)
|
||||
{
|
||||
_buf = buffer;
|
||||
_add = add;
|
||||
_finalize = finalize;
|
||||
_converter = converter;
|
||||
}
|
||||
|
||||
private static bool IsActiveTake(Handler<Action<bool, TOut>> handler)
|
||||
{
|
||||
return handler.IsActive;
|
||||
}
|
||||
|
||||
private static bool IsActivePut((Handler<Action<bool>>, TIn) input)
|
||||
{
|
||||
return input.Item1.IsActive;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tries to put a put into the channel
|
||||
/// </summary>
|
||||
/// <param name="val"></param>
|
||||
/// <param name="handler"></param>
|
||||
/// <returns>(result_type, w)</returns>
|
||||
public override (AsyncResult, bool) Put(TIn val, Handler<Action<bool>> handler)
|
||||
{
|
||||
Monitor.Enter(this);
|
||||
if (_isClosed)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Completed, false);
|
||||
}
|
||||
|
||||
if (_buf != null && !_buf.IsFull && !_takes.IsEmpty)
|
||||
{
|
||||
var put_cb = LockIfActiveCommit(handler);
|
||||
if (put_cb != null)
|
||||
{
|
||||
var is_done = _add(_buf, val);
|
||||
if (!_buf.IsEmpty)
|
||||
{
|
||||
var take_cbs = GetTakersForBuffer();
|
||||
if (is_done)
|
||||
Abort();
|
||||
Monitor.Exit(this);
|
||||
foreach (var action in take_cbs)
|
||||
{
|
||||
Task.Run(action);
|
||||
}
|
||||
return (AsyncResult.Completed, true);
|
||||
}
|
||||
|
||||
if (is_done)
|
||||
Abort();
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Closed, false);
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Canceled, false);
|
||||
}
|
||||
|
||||
var (put_cb2, take_cb) = GetCallbacks(handler, _takes);
|
||||
|
||||
if (put_cb2 != null && take_cb != null)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
Task.Run(() => take_cb(true, _converter(val)));
|
||||
return (AsyncResult.Completed, true);
|
||||
}
|
||||
|
||||
if (_buf != null && !_buf.IsFull)
|
||||
{
|
||||
if (LockIfActiveCommit(handler) != null)
|
||||
{
|
||||
if (_add(_buf, val))
|
||||
{
|
||||
Abort();
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Completed, true);
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Canceled, true);
|
||||
}
|
||||
|
||||
if (handler.IsActive && handler.IsBlockable)
|
||||
{
|
||||
if (_puts.Length >= MAX_QUEUE_SIZE)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
throw new TooManyHanldersException();
|
||||
}
|
||||
_puts.Unshift((handler, val));
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Enqueued, true);
|
||||
}
|
||||
|
||||
public override (AsyncResult, TOut) Take(Handler<Action<bool, TOut>> handler)
|
||||
{
|
||||
Monitor.Enter(this);
|
||||
Cleanup();
|
||||
|
||||
if (_buf != null && !_buf.IsEmpty)
|
||||
{
|
||||
var take_cb = LockIfActiveCommit(handler);
|
||||
if (take_cb != null)
|
||||
{
|
||||
var val = _buf.Remove();
|
||||
var (is_done, cbs) = GetPuttersForBuffer();
|
||||
|
||||
if (is_done)
|
||||
Abort();
|
||||
|
||||
Monitor.Exit(this);
|
||||
|
||||
foreach (var cb in cbs)
|
||||
Task.Run(() => cb(true));
|
||||
|
||||
return (AsyncResult.Completed, val);
|
||||
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Canceled, default);
|
||||
}
|
||||
|
||||
var (take_cb2, put_cb, val2, found) = FindMatchingPut(handler);
|
||||
|
||||
if (take_cb2 != null && put_cb != null)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
Task.Run(() => put_cb(true));
|
||||
return (AsyncResult.Completed, _converter(val2));
|
||||
}
|
||||
|
||||
if (_isClosed)
|
||||
{
|
||||
if (_buf != null && found)
|
||||
_add(_buf, val2);
|
||||
|
||||
var has_val = _buf != null && !_buf.IsEmpty;
|
||||
var take_cb3 = LockIfActiveCommit(handler);
|
||||
|
||||
if (take_cb3 != null)
|
||||
{
|
||||
var val = has_val ? _buf.Remove() : default;
|
||||
Monitor.Exit(this);
|
||||
return has_val ? (AsyncResult.Completed, val) : (AsyncResult.Closed, default);
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Closed, default);
|
||||
}
|
||||
|
||||
if (handler.IsBlockable)
|
||||
{
|
||||
if (_takes.Length >= MAX_QUEUE_SIZE)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
throw new TooManyHanldersException();
|
||||
}
|
||||
|
||||
_takes.Unshift(handler);
|
||||
}
|
||||
Monitor.Exit(this);
|
||||
return (AsyncResult.Enqueued, default);
|
||||
}
|
||||
|
||||
public override bool IsClosed => _isClosed;
|
||||
|
||||
public override void Close()
|
||||
{
|
||||
Monitor.Enter(this);
|
||||
Cleanup();
|
||||
if (_isClosed)
|
||||
{
|
||||
Monitor.Exit(this);
|
||||
return;
|
||||
}
|
||||
|
||||
_isClosed = true;
|
||||
if (_buf != null && _puts.IsEmpty)
|
||||
_finalize(_buf);
|
||||
|
||||
var cbs = _buf == null? new List<Action>() : GetTakersForBuffer();
|
||||
|
||||
while (!_takes.IsEmpty)
|
||||
{
|
||||
var take_cb = LockIfActiveCommit(_takes.Pop());
|
||||
if (take_cb != null)
|
||||
cbs.Add(() => take_cb(false, default));
|
||||
}
|
||||
|
||||
Monitor.Exit(this);
|
||||
|
||||
foreach (var cb in cbs)
|
||||
Task.Run(cb);
|
||||
}
|
||||
|
||||
private (Action<bool, TOut>, Action<bool>, TIn, bool) FindMatchingPut(Handler<Action<bool, TOut>> handler)
|
||||
{
|
||||
while (!_puts.IsEmpty)
|
||||
{
|
||||
var (found, val) = _puts.Peek();
|
||||
var (handler_cb, put_cb, handler_active, put_active) = LockIfActiveCommit(handler, found);
|
||||
|
||||
if (handler_active && put_active)
|
||||
{
|
||||
_puts.Pop();
|
||||
return (handler_cb, put_cb, val, true);
|
||||
}
|
||||
|
||||
if (!put_active)
|
||||
{
|
||||
_puts.Pop();
|
||||
continue;
|
||||
}
|
||||
|
||||
return (null, null, default, false);
|
||||
|
||||
}
|
||||
|
||||
return (null, null, default, false);
|
||||
}
|
||||
|
||||
private (bool, List<Action<bool>>) GetPuttersForBuffer()
|
||||
{
|
||||
List<Action<bool>> acc = new List<Action<bool>>();
|
||||
|
||||
while (!_puts.IsEmpty)
|
||||
{
|
||||
var (putter, val) = _puts.Pop();
|
||||
var cb = LockIfActiveCommit(putter);
|
||||
if (cb != null)
|
||||
{
|
||||
acc.Add(cb);
|
||||
}
|
||||
|
||||
var is_done = _add(_buf, val);
|
||||
if (is_done || _buf.IsFull || _puts.IsEmpty)
|
||||
return (is_done, acc);
|
||||
}
|
||||
|
||||
return (false, acc);
|
||||
}
|
||||
|
||||
private void Cleanup()
|
||||
{
|
||||
_takes.Cleanup(IsActiveTake);
|
||||
_puts.Cleanup(IsActivePut);
|
||||
}
|
||||
|
||||
private (T1, T2) GetCallbacks<T1, T2>(Handler<T1> handler, RingBuffer<Handler<T2>> queue)
|
||||
{
|
||||
while (!queue.IsEmpty)
|
||||
{
|
||||
var found = queue.Peek();
|
||||
var (handler_cb, found_cb, handler_valid, found_valid) = LockIfActiveCommit(handler, found);
|
||||
if (handler_valid && found_valid)
|
||||
{
|
||||
queue.Pop();
|
||||
return (handler_cb, found_cb);
|
||||
}
|
||||
|
||||
if (handler_valid)
|
||||
{
|
||||
queue.Pop();
|
||||
}
|
||||
else
|
||||
{
|
||||
return (default, default);
|
||||
}
|
||||
}
|
||||
|
||||
return (default, default);
|
||||
}
|
||||
|
||||
private void Abort()
|
||||
{
|
||||
while (!_puts.IsEmpty)
|
||||
{
|
||||
var (handler, val) = _puts.Pop();
|
||||
var put_cb = LockIfActiveCommit(handler);
|
||||
if (put_cb != null)
|
||||
{
|
||||
Task.Run(() => put_cb(false));
|
||||
}
|
||||
}
|
||||
_puts.Cleanup(x => false);
|
||||
Close();
|
||||
}
|
||||
|
||||
private List<Action> GetTakersForBuffer()
|
||||
{
|
||||
List<Action> ret = new List<Action>();
|
||||
while (!_buf.IsEmpty && !_takes.IsEmpty)
|
||||
{
|
||||
var taker = _takes.Pop();
|
||||
var take_cp = LockIfActiveCommit(taker);
|
||||
if (take_cp != null)
|
||||
{
|
||||
var val = _buf.Remove();
|
||||
ret.Add(() => take_cp(true, val));
|
||||
}
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
||||
private static T LockIfActiveCommit<T>(Handler<T> handler)
|
||||
{
|
||||
lock (handler)
|
||||
{
|
||||
return handler.IsActive ? handler.Commit() : default;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
[MethodImpl(MethodImplOptions.AggressiveInlining)]
|
||||
private static (T1, T2, bool, bool) LockIfActiveCommit<T1, T2>(Handler<T1> handler1, Handler<T2> handler2)
|
||||
{
|
||||
if (handler1.LockId < handler2.LockId)
|
||||
{
|
||||
Monitor.Enter(handler1);
|
||||
Monitor.Enter(handler2);
|
||||
}
|
||||
else
|
||||
{
|
||||
Monitor.Enter(handler2);
|
||||
Monitor.Enter(handler1);
|
||||
}
|
||||
|
||||
if (handler1.IsActive && handler2.IsActive)
|
||||
{
|
||||
var ret1 = (handler1.Commit(), handler2.Commit(), true, true);
|
||||
Monitor.Exit(handler1);
|
||||
Monitor.Exit(handler2);
|
||||
return ret1;
|
||||
}
|
||||
|
||||
var ret2 = (default(T1), default(T2), handler1.IsActive, handler2.IsActive);
|
||||
Monitor.Exit(handler1);
|
||||
Monitor.Exit(handler2);
|
||||
return ret2;
|
||||
}
|
||||
|
||||
public class TooManyHanldersException : Exception
|
||||
{
|
||||
public override string ToString()
|
||||
{
|
||||
return $"No more than {MAX_QUEUE_SIZE} pending operations allowed on a single channel.";
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
162
Wabbajack.Common.CSP/PIpelines.cs
Normal file
162
Wabbajack.Common.CSP/PIpelines.cs
Normal file
@ -0,0 +1,162 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Reactive.Subjects;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public static class Pipelines
|
||||
{
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Creates a pipeline that takes items from `from` transforms them with the pipeline given by `transform` and puts
|
||||
/// the resulting values onto `to`. The pipeline may create 0 or more items for every input item and they will be
|
||||
/// spooled onto `to` in a undefined order. `n` determines how many parallel tasks will be running at once. Each of
|
||||
/// these tasks maintains its own transformation pipeline, so `transform` will be called once for every `n`. Completing
|
||||
/// a `transform` pipeline has no effect.
|
||||
/// </summary>
|
||||
/// <typeparam name="TInSrc"></typeparam>
|
||||
/// <typeparam name="TOutSrc"></typeparam>
|
||||
/// <typeparam name="TInDest"></typeparam>
|
||||
/// <typeparam name="TOutDest"></typeparam>
|
||||
/// <param name="from"></param>
|
||||
/// <param name="parallelism"></param>
|
||||
/// <param name="to"></param>
|
||||
/// <param name="transform"></param>
|
||||
/// <param name="propagateClose"></param>
|
||||
/// <returns></returns>
|
||||
public static async Task UnorderedPipeline<TIn, TOut>(
|
||||
this IReadPort<TIn> from,
|
||||
int parallelism,
|
||||
IWritePort<TOut> to,
|
||||
Func<IObservable<TIn>, IObservable<TOut>> transform,
|
||||
bool propagateClose = true)
|
||||
{
|
||||
async Task Pump()
|
||||
{
|
||||
var pipeline = new Subject<TIn>();
|
||||
var buffer = new List<TOut>();
|
||||
var dest = transform(pipeline);
|
||||
dest.Subscribe(itm => buffer.Add(itm));
|
||||
while (true)
|
||||
{
|
||||
var (is_open, tval) = await from.Take();
|
||||
if (is_open)
|
||||
{
|
||||
pipeline.OnNext(tval);
|
||||
foreach (var pval in buffer)
|
||||
{
|
||||
var is_put_open = await to.Put(pval);
|
||||
if (is_put_open) continue;
|
||||
if (propagateClose) @from.Close();
|
||||
return;
|
||||
}
|
||||
buffer.Clear();
|
||||
}
|
||||
else
|
||||
{
|
||||
pipeline.OnCompleted();
|
||||
if (buffer.Count > 0)
|
||||
{
|
||||
foreach (var pval in buffer)
|
||||
if (!await to.Put(pval))
|
||||
break;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
await Task.WhenAll(Enumerable.Range(0, parallelism)
|
||||
.Select(idx => Task.Run(Pump)));
|
||||
|
||||
if (propagateClose)
|
||||
{
|
||||
from.Close();
|
||||
to.Close();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public static async Task UnorderedPipeline<TIn, TOut>(
|
||||
this IReadPort<TIn> from,
|
||||
IWritePort<TOut> to,
|
||||
Func<TIn, Task<TOut>> f,
|
||||
bool propagateClose = true)
|
||||
{
|
||||
await UnorderedPipeline(from, Environment.ProcessorCount, to, f, propagateClose);
|
||||
}
|
||||
|
||||
public static async Task UnorderedPipeline<TIn, TOut>(
|
||||
this IReadPort<TIn> from,
|
||||
int parallelism,
|
||||
IWritePort<TOut> to,
|
||||
Func<TIn, Task<TOut>> f,
|
||||
bool propagateClose = true)
|
||||
{
|
||||
async Task Pump()
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var (is_open, job) = await from.Take();
|
||||
if (!is_open) break;
|
||||
|
||||
var putIsOpen = await to.Put(await f(job));
|
||||
if (!putIsOpen) return;
|
||||
}
|
||||
}
|
||||
|
||||
await Task.WhenAll(Enumerable.Range(0, parallelism)
|
||||
.Select(idx => Task.Run(Pump)));
|
||||
|
||||
if (propagateClose)
|
||||
{
|
||||
from.Close();
|
||||
to.Close();
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public static async Task UnorderedThreadedPipeline<TIn, TOut>(
|
||||
this IReadPort<TIn> from,
|
||||
int parallelism,
|
||||
IWritePort<TOut> to,
|
||||
Func<TIn, TOut> f,
|
||||
bool propagateClose = true)
|
||||
{
|
||||
Task Pump()
|
||||
{
|
||||
var tcs = new TaskCompletionSource<bool>();
|
||||
|
||||
var th = new Thread(() =>
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var (is_open, job) = from.Take().Result;
|
||||
if (!is_open) break;
|
||||
|
||||
var putIsOpen = to.Put(f(job)).Result;
|
||||
if (!putIsOpen) return;
|
||||
}
|
||||
tcs.SetResult(true);
|
||||
}) {Priority = ThreadPriority.BelowNormal};
|
||||
th.Start();
|
||||
return tcs.Task;
|
||||
}
|
||||
|
||||
await Task.WhenAll(Enumerable.Range(0, parallelism)
|
||||
.Select(idx => Task.Run(Pump)));
|
||||
|
||||
if (propagateClose)
|
||||
{
|
||||
from.Close();
|
||||
to.Close();
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
36
Wabbajack.Common.CSP/Properties/AssemblyInfo.cs
Normal file
36
Wabbajack.Common.CSP/Properties/AssemblyInfo.cs
Normal file
@ -0,0 +1,36 @@
|
||||
using System.Reflection;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
// General Information about an assembly is controlled through the following
|
||||
// set of attributes. Change these attribute values to modify the information
|
||||
// associated with an assembly.
|
||||
[assembly: AssemblyTitle("Wabbajack.CSP")]
|
||||
[assembly: AssemblyDescription("")]
|
||||
[assembly: AssemblyConfiguration("")]
|
||||
[assembly: AssemblyCompany("")]
|
||||
[assembly: AssemblyProduct("Wabbajack.CSP")]
|
||||
[assembly: AssemblyCopyright("Copyright © 2019")]
|
||||
[assembly: AssemblyTrademark("")]
|
||||
[assembly: AssemblyCulture("")]
|
||||
|
||||
// Setting ComVisible to false makes the types in this assembly not visible
|
||||
// to COM components. If you need to access a type in this assembly from
|
||||
// COM, set the ComVisible attribute to true on that type.
|
||||
[assembly: ComVisible(false)]
|
||||
|
||||
// The following GUID is for the ID of the typelib if this project is exposed to COM
|
||||
[assembly: Guid("9e69bc98-1512-4977-b683-6e7e5292c0b8")]
|
||||
|
||||
// Version information for an assembly consists of the following four values:
|
||||
//
|
||||
// Major Version
|
||||
// Minor Version
|
||||
// Build Number
|
||||
// Revision
|
||||
//
|
||||
// You can specify all the values or you can default the Build and Revision Numbers
|
||||
// by using the '*' as shown below:
|
||||
// [assembly: AssemblyVersion("1.0.*")]
|
||||
[assembly: AssemblyVersion("1.0.0.0")]
|
||||
[assembly: AssemblyFileVersion("1.0.0.0")]
|
||||
42
Wabbajack.Common.CSP/PutTaskHandler.cs
Normal file
42
Wabbajack.Common.CSP/PutTaskHandler.cs
Normal file
@ -0,0 +1,42 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
class PutTaskHandler<T> : Handler<Action<bool>>
|
||||
{
|
||||
private readonly bool _blockable;
|
||||
private TaskCompletionSource<bool> _tcs = new TaskCompletionSource<bool>();
|
||||
|
||||
public PutTaskHandler(bool blockable = true)
|
||||
{
|
||||
_blockable = blockable;
|
||||
}
|
||||
|
||||
public TaskCompletionSource<bool> TaskCompletionSource
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_tcs == null)
|
||||
_tcs = new TaskCompletionSource<bool>();
|
||||
return _tcs;
|
||||
}
|
||||
}
|
||||
|
||||
public bool IsActive => true;
|
||||
public bool IsBlockable => _blockable;
|
||||
public uint LockId => 0;
|
||||
public Action<bool> Commit()
|
||||
{
|
||||
return Handle;
|
||||
}
|
||||
|
||||
private void Handle(bool val)
|
||||
{
|
||||
TaskCompletionSource.SetResult(val);
|
||||
}
|
||||
}
|
||||
}
|
||||
113
Wabbajack.Common.CSP/RingBuffer.cs
Normal file
113
Wabbajack.Common.CSP/RingBuffer.cs
Normal file
@ -0,0 +1,113 @@
|
||||
using System;
|
||||
using System.Collections;
|
||||
using System.Collections.Generic;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public class RingBuffer<T> : IEnumerable<T>
|
||||
{
|
||||
private int _size;
|
||||
private int _length;
|
||||
private int _tail;
|
||||
private int _head;
|
||||
private T[] _arr;
|
||||
|
||||
public RingBuffer(int size = 8)
|
||||
{
|
||||
_size = size;
|
||||
_arr = new T[size];
|
||||
_tail = 0;
|
||||
_length = 0;
|
||||
_head = 0;
|
||||
}
|
||||
|
||||
public T Pop()
|
||||
{
|
||||
if (_length == 0) return default;
|
||||
var val = _arr[_tail];
|
||||
_arr[_tail] = default;
|
||||
_tail = (_tail + 1) % _size;
|
||||
_length -= 1;
|
||||
return val;
|
||||
}
|
||||
|
||||
public T Peek()
|
||||
{
|
||||
return _length == 0 ? default : _arr[_tail];
|
||||
}
|
||||
|
||||
public void Unshift(T x)
|
||||
{
|
||||
_arr[_head] = x;
|
||||
_head = (_head + 1) % _size;
|
||||
_length += 1;
|
||||
}
|
||||
|
||||
public void UnboundedUnshift(T x)
|
||||
{
|
||||
if (_length == _size)
|
||||
Resize();
|
||||
Unshift(x);
|
||||
}
|
||||
|
||||
public bool IsEmpty => _length == 0;
|
||||
public int Length => _length;
|
||||
|
||||
private void Resize()
|
||||
{
|
||||
var new_arr_size = _size * 2;
|
||||
var new_arr = new T[new_arr_size];
|
||||
if (_tail < _head)
|
||||
{
|
||||
Array.Copy(_arr, _tail, new_arr, 0, _length);
|
||||
_tail = 0;
|
||||
_head = _length;
|
||||
_arr = new_arr;
|
||||
_size = new_arr_size;
|
||||
}
|
||||
else if (_tail > _head)
|
||||
{
|
||||
Array.Copy(_arr, _tail, new_arr, 0, _length - _tail);
|
||||
Array.Copy(_arr, 0, new_arr, (_length - _tail), _head);
|
||||
_tail = 0;
|
||||
_head = _length;
|
||||
_arr = new_arr;
|
||||
_size = new_arr_size;
|
||||
}
|
||||
else
|
||||
{
|
||||
_tail = 0;
|
||||
_head = 0;
|
||||
_arr = new_arr;
|
||||
_size = new_arr_size;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Filers out all items where should_keep(itm) returns false
|
||||
/// </summary>
|
||||
/// <param name="should_keep"></param>
|
||||
public void Cleanup(Func<T, bool> should_keep)
|
||||
{
|
||||
for (var idx = 0; idx < _length; idx++)
|
||||
{
|
||||
var v = Pop();
|
||||
if (should_keep(v))
|
||||
{
|
||||
Unshift(v);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public IEnumerator<T> GetEnumerator()
|
||||
{
|
||||
while (!IsEmpty)
|
||||
yield return Pop();
|
||||
}
|
||||
|
||||
IEnumerator IEnumerable.GetEnumerator()
|
||||
{
|
||||
return GetEnumerator();
|
||||
}
|
||||
}
|
||||
}
|
||||
68
Wabbajack.Common.CSP/RxBuffer.cs
Normal file
68
Wabbajack.Common.CSP/RxBuffer.cs
Normal file
@ -0,0 +1,68 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Reactive.Linq;
|
||||
using System.Reactive.Subjects;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public class RxBuffer<TIn, TOut> : LinkedList<TOut>, IBuffer<TOut>
|
||||
{
|
||||
private Subject<TIn> _inputSubject;
|
||||
private IObservable<TOut> _outputObservable;
|
||||
private bool _completed;
|
||||
private int _maxSize;
|
||||
|
||||
public RxBuffer(int size, Func<IObservable<TIn>, IObservable<TOut>> transform) : base()
|
||||
{
|
||||
_maxSize = size;
|
||||
_inputSubject = new Subject<TIn>();
|
||||
_outputObservable = transform(_inputSubject);
|
||||
_outputObservable.Subscribe(itm => AddFirst(itm), () => {
|
||||
_completed = true;
|
||||
});
|
||||
}
|
||||
|
||||
public bool TransformAdd(TIn val)
|
||||
{
|
||||
_inputSubject.OnNext(val);
|
||||
return _completed;
|
||||
}
|
||||
|
||||
public static bool TransformAdd(IBuffer<TOut> buf, TIn itm)
|
||||
{
|
||||
return ((RxBuffer<TIn, TOut>) buf).TransformAdd(itm);
|
||||
}
|
||||
|
||||
public void Finalize()
|
||||
{
|
||||
_inputSubject.OnCompleted();
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
throw new NotImplementedException();
|
||||
}
|
||||
|
||||
public static void Finalize(IBuffer<TOut> buf)
|
||||
{
|
||||
((RxBuffer<TIn, TOut>)buf).Finalize();
|
||||
}
|
||||
|
||||
public bool IsFull => Count >= _maxSize;
|
||||
public bool IsEmpty => Count == 0;
|
||||
public TOut Remove()
|
||||
{
|
||||
var ret = Last.Value;
|
||||
RemoveLast();
|
||||
return ret;
|
||||
}
|
||||
|
||||
public void Add(TOut itm)
|
||||
{
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
50
Wabbajack.Common.CSP/TakeTaskHandler.cs
Normal file
50
Wabbajack.Common.CSP/TakeTaskHandler.cs
Normal file
@ -0,0 +1,50 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Text;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common.CSP
|
||||
{
|
||||
public class TakeTaskHandler<T> : Handler<Action<bool, T>>
|
||||
{
|
||||
private readonly bool _blockable;
|
||||
private TaskCompletionSource<(bool, T)> _tcs;
|
||||
|
||||
public TakeTaskHandler(TaskCompletionSource<T> tcs = null, bool blockable = true)
|
||||
{
|
||||
_blockable = blockable;
|
||||
}
|
||||
|
||||
public TaskCompletionSource<(bool, T)> TaskCompletionSource
|
||||
{
|
||||
get
|
||||
{
|
||||
if (_tcs == null)
|
||||
{
|
||||
var new_tcs = new TaskCompletionSource<(bool, T)>();
|
||||
Interlocked.CompareExchange(ref _tcs, new_tcs, null);
|
||||
}
|
||||
|
||||
return _tcs;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
public bool IsActive => true;
|
||||
public bool IsBlockable => _blockable;
|
||||
public uint LockId => 0;
|
||||
public Task<(bool, T)> Task => TaskCompletionSource.Task;
|
||||
public Action<bool, T> Commit()
|
||||
{
|
||||
return Handle;
|
||||
}
|
||||
|
||||
private void Handle(bool is_open, T a)
|
||||
{
|
||||
TaskCompletionSource.SetResult((is_open, a));
|
||||
}
|
||||
}
|
||||
}
|
||||
105
Wabbajack.Common.CSP/Wabbajack.Common.CSP.csproj
Normal file
105
Wabbajack.Common.CSP/Wabbajack.Common.CSP.csproj
Normal file
@ -0,0 +1,105 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
|
||||
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
|
||||
<PropertyGroup>
|
||||
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
|
||||
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
|
||||
<ProjectGuid>{9E69BC98-1512-4977-B683-6E7E5292C0B8}</ProjectGuid>
|
||||
<OutputType>Library</OutputType>
|
||||
<AppDesignerFolder>Properties</AppDesignerFolder>
|
||||
<RootNamespace>Wabbajack.CSP</RootNamespace>
|
||||
<AssemblyName>Wabbajack.CSP</AssemblyName>
|
||||
<TargetFrameworkVersion>v4.7.2</TargetFrameworkVersion>
|
||||
<FileAlignment>512</FileAlignment>
|
||||
<Deterministic>true</Deterministic>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
|
||||
<DebugSymbols>true</DebugSymbols>
|
||||
<DebugType>full</DebugType>
|
||||
<Optimize>false</Optimize>
|
||||
<OutputPath>bin\Debug\</OutputPath>
|
||||
<DefineConstants>DEBUG;TRACE</DefineConstants>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<WarningLevel>4</WarningLevel>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<Optimize>true</Optimize>
|
||||
<OutputPath>bin\Release\</OutputPath>
|
||||
<DefineConstants>TRACE</DefineConstants>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<WarningLevel>4</WarningLevel>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
|
||||
<DebugSymbols>true</DebugSymbols>
|
||||
<OutputPath>bin\x64\Debug\</OutputPath>
|
||||
<DefineConstants>DEBUG;TRACE</DefineConstants>
|
||||
<DebugType>full</DebugType>
|
||||
<PlatformTarget>x64</PlatformTarget>
|
||||
<LangVersion>7.3</LangVersion>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
</PropertyGroup>
|
||||
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
|
||||
<OutputPath>bin\x64\Release\</OutputPath>
|
||||
<DefineConstants>TRACE</DefineConstants>
|
||||
<Optimize>true</Optimize>
|
||||
<DebugType>pdbonly</DebugType>
|
||||
<PlatformTarget>x64</PlatformTarget>
|
||||
<LangVersion>7.3</LangVersion>
|
||||
<ErrorReport>prompt</ErrorReport>
|
||||
<CodeAnalysisRuleSet>MinimumRecommendedRules.ruleset</CodeAnalysisRuleSet>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Reference Include="System" />
|
||||
<Reference Include="System.Core" />
|
||||
<Reference Include="System.Reactive, Version=4.2.0.0, Culture=neutral, PublicKeyToken=94bc3704cddfc263, processorArchitecture=MSIL">
|
||||
<HintPath>..\packages\System.Reactive.4.2.0\lib\net46\System.Reactive.dll</HintPath>
|
||||
</Reference>
|
||||
<Reference Include="System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
|
||||
<HintPath>..\packages\System.Runtime.CompilerServices.Unsafe.4.5.2\lib\netstandard2.0\System.Runtime.CompilerServices.Unsafe.dll</HintPath>
|
||||
</Reference>
|
||||
<Reference Include="System.Threading.Tasks.Extensions, Version=4.2.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
|
||||
<HintPath>..\packages\System.Threading.Tasks.Extensions.4.5.3\lib\netstandard2.0\System.Threading.Tasks.Extensions.dll</HintPath>
|
||||
</Reference>
|
||||
<Reference Include="System.ValueTuple, Version=4.0.3.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
|
||||
<HintPath>..\packages\System.ValueTuple.4.5.0\lib\net47\System.ValueTuple.dll</HintPath>
|
||||
</Reference>
|
||||
<Reference Include="System.Windows" />
|
||||
<Reference Include="System.Windows.Forms" />
|
||||
<Reference Include="System.Xml.Linq" />
|
||||
<Reference Include="System.Data.DataSetExtensions" />
|
||||
<Reference Include="Microsoft.CSharp" />
|
||||
<Reference Include="System.Data" />
|
||||
<Reference Include="System.Net.Http" />
|
||||
<Reference Include="System.Xml" />
|
||||
<Reference Include="WindowsBase" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Compile Include="AChannel.cs" />
|
||||
<Compile Include="AsyncResult.cs" />
|
||||
<Compile Include="Channel.cs" />
|
||||
<Compile Include="Extensions.cs" />
|
||||
<Compile Include="EnumeratorBuffer.cs" />
|
||||
<Compile Include="FixedSizeBuffer.cs" />
|
||||
<Compile Include="Handler.cs" />
|
||||
<Compile Include="IBuffer.cs" />
|
||||
<Compile Include="IChannel.cs" />
|
||||
<Compile Include="ICloseable.cs" />
|
||||
<Compile Include="IReadPort.cs" />
|
||||
<Compile Include="IWritePort.cs" />
|
||||
<Compile Include="ManyToManyChannel.cs" />
|
||||
<Compile Include="PIpelines.cs" />
|
||||
<Compile Include="PutTaskHandler.cs" />
|
||||
<Compile Include="RingBuffer.cs" />
|
||||
<Compile Include="RxBuffer.cs" />
|
||||
<Compile Include="TakeTaskHandler.cs" />
|
||||
<Compile Include="Properties\AssemblyInfo.cs" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<None Include="CSP Readme.md" />
|
||||
<None Include="packages.config" />
|
||||
</ItemGroup>
|
||||
<ItemGroup />
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
|
||||
</Project>
|
||||
7
Wabbajack.Common.CSP/packages.config
Normal file
7
Wabbajack.Common.CSP/packages.config
Normal file
@ -0,0 +1,7 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<packages>
|
||||
<package id="System.Reactive" version="4.2.0" targetFramework="net472" />
|
||||
<package id="System.Runtime.CompilerServices.Unsafe" version="4.5.2" targetFramework="net472" />
|
||||
<package id="System.Threading.Tasks.Extensions" version="4.5.3" targetFramework="net472" />
|
||||
<package id="System.ValueTuple" version="4.5.0" targetFramework="net472" />
|
||||
</packages>
|
||||
42
Wabbajack.Common/ChannelStreams.cs
Normal file
42
Wabbajack.Common/ChannelStreams.cs
Normal file
@ -0,0 +1,42 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Channels;
|
||||
using System.Threading.Tasks;
|
||||
|
||||
namespace Wabbajack.Common
|
||||
{
|
||||
public static class ChannelStreams
|
||||
{
|
||||
public static async Task IntoChannel<T>(this IEnumerable<T> coll, Channel<T> dest, bool closeAtEnd = false)
|
||||
{
|
||||
foreach (var itm in coll)
|
||||
{
|
||||
await dest.Writer.WriteAsync(itm);
|
||||
}
|
||||
|
||||
if (closeAtEnd)
|
||||
dest.Writer.Complete();
|
||||
}
|
||||
|
||||
public static Channel<T> AsChannel<T>(this IEnumerable<T> coll)
|
||||
{
|
||||
var chan = Channel.CreateUnbounded<T>();
|
||||
coll.IntoChannel(chan, true);
|
||||
return chan;
|
||||
}
|
||||
|
||||
public static async Task<IEnumerable<T>> ToIEnumerable<T>(this Channel<T> src)
|
||||
{
|
||||
var buffer = new List<T>();
|
||||
while (true)
|
||||
{
|
||||
var result = await src.Reader.ReadAsync();
|
||||
buffer.Add(result);
|
||||
}
|
||||
|
||||
return buffer;
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -19,7 +19,7 @@ namespace Wabbajack.Common
|
||||
|
||||
public static HashSet<string> SupportedArchives = new HashSet<string> {".zip", ".rar", ".7z", ".7zip", ".fomod", ".omod"};
|
||||
|
||||
public static HashSet<string> SupportedBSAs = new HashSet<string> {".bsa", ".ba2"};
|
||||
public static HashSet<string> SupportedBSAs = new HashSet<string> {".bsa", ".ba2", ".BA2"};
|
||||
|
||||
public static HashSet<string> ConfigFileExtensions = new HashSet<string> {".json", ".ini", ".yml"};
|
||||
public static HashSet<string> ESPFileExtensions = new HashSet<string>() { ".esp", ".esm", ".esl"};
|
||||
|
||||
@ -2,10 +2,12 @@
|
||||
using System.Diagnostics;
|
||||
using System.Linq;
|
||||
using System.Reflection;
|
||||
using System.Threading.Tasks;
|
||||
using Alphaleonis.Win32.Filesystem;
|
||||
using Compression.BSA;
|
||||
using ICSharpCode.SharpZipLib.GZip;
|
||||
using OMODFramework;
|
||||
using Wabbajack.Common.CSP;
|
||||
|
||||
namespace Wabbajack.Common
|
||||
{
|
||||
@ -31,18 +33,16 @@ namespace Wabbajack.Common
|
||||
}
|
||||
|
||||
|
||||
public static void ExtractAll(string source, string dest)
|
||||
public static async Task ExtractAll(string source, string dest)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (Consts.SupportedBSAs.Any(b => source.ToLower().EndsWith(b)))
|
||||
ExtractAllWithBSA(source, dest);
|
||||
else if (source.EndsWith(".exe"))
|
||||
ExtractAllWithInno(source, dest);
|
||||
await ExtractAllWithBSA(source, dest);
|
||||
else if (source.EndsWith(".omod"))
|
||||
ExtractAllWithOMOD(source, dest);
|
||||
await ExtractAllWithOMOD(source, dest);
|
||||
else
|
||||
ExtractAllWith7Zip(source, dest);
|
||||
await ExtractAllWith7Zip(source, dest);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@ -51,39 +51,48 @@ namespace Wabbajack.Common
|
||||
}
|
||||
}
|
||||
|
||||
private static void ExtractAllWithOMOD(string source, string dest)
|
||||
private static Task ExtractAllWithOMOD(string source, string dest)
|
||||
{
|
||||
Utils.Log($"Extracting {Path.GetFileName(source)}");
|
||||
var f = new Framework();
|
||||
f.SetTempDirectory(dest);
|
||||
var omod = new OMOD(source, ref f);
|
||||
omod.ExtractDataFiles();
|
||||
omod.ExtractPlugins();
|
||||
return CSPExtensions.ThreadedTask(() =>
|
||||
{
|
||||
Utils.Log($"Extracting {Path.GetFileName(source)}");
|
||||
var f = new Framework();
|
||||
f.SetTempDirectory(dest);
|
||||
var omod = new OMOD(source, ref f);
|
||||
omod.ExtractDataFiles();
|
||||
omod.ExtractPlugins();
|
||||
return dest;
|
||||
});
|
||||
}
|
||||
|
||||
private static void ExtractAllWithBSA(string source, string dest)
|
||||
private static async Task ExtractAllWithBSA(string source, string dest)
|
||||
{
|
||||
try
|
||||
{
|
||||
using (var arch = BSADispatch.OpenRead(source))
|
||||
using (var arch = await BSADispatch.OpenRead(source))
|
||||
{
|
||||
arch.Files.PMap(f =>
|
||||
{
|
||||
var path = f.Path;
|
||||
if (f.Path.StartsWith("\\"))
|
||||
path = f.Path.Substring(1);
|
||||
Utils.Status($"Extracting {path}");
|
||||
var out_path = Path.Combine(dest, path);
|
||||
var parent = Path.GetDirectoryName(out_path);
|
||||
await arch.Files.ToChannel()
|
||||
.UnorderedPipeline(
|
||||
Channel.CreateSink<IFile>(),
|
||||
async f =>
|
||||
{
|
||||
var path = f.Path;
|
||||
if (f.Path.StartsWith("\\"))
|
||||
path = f.Path.Substring(1);
|
||||
Utils.Status($"Extracting {path}");
|
||||
var out_path = Path.Combine(dest, path);
|
||||
var parent = Path.GetDirectoryName(out_path);
|
||||
|
||||
if (!Directory.Exists(parent))
|
||||
Directory.CreateDirectory(parent);
|
||||
if (!Directory.Exists(parent))
|
||||
Directory.CreateDirectory(parent);
|
||||
|
||||
using (var fs = File.OpenWrite(out_path))
|
||||
{
|
||||
f.CopyDataTo(fs);
|
||||
}
|
||||
});
|
||||
using (var fs = File.OpenWrite(out_path))
|
||||
{
|
||||
await f.CopyDataToAsync(fs);
|
||||
}
|
||||
|
||||
return f;
|
||||
});
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
@ -93,7 +102,7 @@ namespace Wabbajack.Common
|
||||
}
|
||||
}
|
||||
|
||||
private static void ExtractAllWith7Zip(string source, string dest)
|
||||
private static async Task ExtractAllWith7Zip(string source, string dest)
|
||||
{
|
||||
Utils.Log($"Extracting {Path.GetFileName(source)}");
|
||||
|
||||
@ -128,73 +137,14 @@ namespace Wabbajack.Common
|
||||
{
|
||||
while (!p.HasExited)
|
||||
{
|
||||
var line = p.StandardOutput.ReadLine();
|
||||
var line = await p.StandardOutput.ReadLineAsync();
|
||||
if (line == null)
|
||||
break;
|
||||
var percent = 0;
|
||||
if (line.Length > 4 && line[3] == '%')
|
||||
{
|
||||
int.TryParse(line.Substring(0, 3), out percent);
|
||||
Utils.Status($"Extracting {name} - {line.Trim()}", percent);
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception)
|
||||
{
|
||||
}
|
||||
|
||||
p.WaitForExit();
|
||||
if (p.ExitCode != 0)
|
||||
{
|
||||
Utils.Log(p.StandardOutput.ReadToEnd());
|
||||
Utils.Log($"Extraction error extracting {source}");
|
||||
}
|
||||
}
|
||||
|
||||
private static void ExtractAllWithInno(string source, string dest)
|
||||
{
|
||||
Utils.Log($"Extracting {Path.GetFileName(source)}");
|
||||
|
||||
var info = new ProcessStartInfo
|
||||
{
|
||||
FileName = "innounp.exe",
|
||||
Arguments = $"-x -y -b -d\"{dest}\" \"{source}\"",
|
||||
RedirectStandardError = true,
|
||||
RedirectStandardInput = true,
|
||||
RedirectStandardOutput = true,
|
||||
UseShellExecute = false,
|
||||
CreateNoWindow = true
|
||||
};
|
||||
|
||||
var p = new Process
|
||||
{
|
||||
StartInfo = info
|
||||
};
|
||||
|
||||
p.Start();
|
||||
ChildProcessTracker.AddProcess(p);
|
||||
try
|
||||
{
|
||||
p.PriorityClass = ProcessPriorityClass.BelowNormal;
|
||||
}
|
||||
catch (Exception)
|
||||
{
|
||||
}
|
||||
|
||||
var name = Path.GetFileName(source);
|
||||
try
|
||||
{
|
||||
while (!p.HasExited)
|
||||
{
|
||||
var line = p.StandardOutput.ReadLine();
|
||||
if (line == null)
|
||||
break;
|
||||
var percent = 0;
|
||||
if (line.Length > 4 && line[3] == '%')
|
||||
{
|
||||
int.TryParse(line.Substring(0, 3), out percent);
|
||||
Utils.Status($"Extracting {name} - {line.Trim()}", percent);
|
||||
}
|
||||
|
||||
if (line.Length <= 4 || line[3] != '%') continue;
|
||||
|
||||
int.TryParse(line.Substring(0, 3), out var percent);
|
||||
Utils.Status($"Extracting {name} - {line.Trim()}", percent);
|
||||
}
|
||||
}
|
||||
catch (Exception)
|
||||
@ -219,11 +169,5 @@ namespace Wabbajack.Common
|
||||
v = v.ToLower();
|
||||
return Consts.SupportedArchives.Contains(v) || Consts.SupportedBSAs.Contains(v);
|
||||
}
|
||||
|
||||
public class Entry
|
||||
{
|
||||
public string Name;
|
||||
public ulong Size;
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -497,24 +497,7 @@ namespace Wabbajack.Common
|
||||
|
||||
public static string ExceptionToString(this Exception ex)
|
||||
{
|
||||
var sb = new StringBuilder();
|
||||
while (ex != null)
|
||||
{
|
||||
sb.AppendLine(ex.Message);
|
||||
var st = new StackTrace(ex, true);
|
||||
foreach (var frame in st.GetFrames())
|
||||
sb.AppendLine(
|
||||
$"{frame.GetFileName()}:{frame.GetMethod().Name}:{frame.GetFileLineNumber()}:{frame.GetFileColumnNumber()}");
|
||||
ex = ex.InnerException;
|
||||
}
|
||||
|
||||
|
||||
return sb.ToString();
|
||||
}
|
||||
|
||||
public static void CrashDump(Exception e)
|
||||
{
|
||||
File.WriteAllText($"{DateTime.Now.ToString("yyyyMMddTHHmmss_crash_log.txt")}", ExceptionToString(e));
|
||||
return ex.ToString();
|
||||
}
|
||||
|
||||
public static IEnumerable<T> DistinctBy<T, V>(this IEnumerable<T> vs, Func<T, V> select)
|
||||
|
||||
@ -115,6 +115,10 @@
|
||||
<Project>{ff5d892f-8ff4-44fc-8f7f-cd58f307ad1b}</Project>
|
||||
<Name>Compression.BSA</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Common.CSP\Wabbajack.Common.CSP.csproj">
|
||||
<Project>{9e69bc98-1512-4977-b683-6e7e5292c0b8}</Project>
|
||||
<Name>Wabbajack.Common.CSP</Name>
|
||||
</ProjectReference>
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Content Include="7z.dll" />
|
||||
@ -149,5 +153,6 @@
|
||||
<Version>8.0.0</Version>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
<ItemGroup />
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
|
||||
</Project>
|
||||
@ -75,7 +75,7 @@ namespace Wabbajack.Lib.CompilationSteps
|
||||
}
|
||||
|
||||
CreateBSA directive;
|
||||
using (var bsa = BSADispatch.OpenRead(source.AbsolutePath))
|
||||
using (var bsa = BSADispatch.OpenRead(source.AbsolutePath).Result)
|
||||
{
|
||||
directive = new CreateBSA
|
||||
{
|
||||
|
||||
@ -11,6 +11,7 @@ using System.Linq;
|
||||
using System.Reflection;
|
||||
using System.Text;
|
||||
using System.Text.RegularExpressions;
|
||||
using System.Threading.Tasks;
|
||||
using VFS;
|
||||
using Wabbajack.Common;
|
||||
using Wabbajack.Lib.CompilationSteps;
|
||||
@ -449,7 +450,7 @@ namespace Wabbajack.Lib
|
||||
using (var output = new MemoryStream())
|
||||
{
|
||||
var a = origin.ReadAll();
|
||||
var b = LoadDataForTo(entry.To, absolute_paths);
|
||||
var b = LoadDataForTo(entry.To, absolute_paths).Result;
|
||||
Utils.CreatePatch(a, b, output);
|
||||
entry.PatchID = IncludeFile(output.ToArray());
|
||||
var file_size = File.GetSize(Path.Combine(ModListOutputFolder, entry.PatchID));
|
||||
@ -459,7 +460,7 @@ namespace Wabbajack.Lib
|
||||
}
|
||||
}
|
||||
|
||||
private byte[] LoadDataForTo(string to, Dictionary<string, string> absolute_paths)
|
||||
private async Task<byte[]> LoadDataForTo(string to, Dictionary<string, string> absolute_paths)
|
||||
{
|
||||
if (absolute_paths.TryGetValue(to, out var absolute))
|
||||
return File.ReadAllBytes(absolute);
|
||||
@ -469,13 +470,13 @@ namespace Wabbajack.Lib
|
||||
var bsa_id = to.Split('\\')[1];
|
||||
var bsa = InstallDirectives.OfType<CreateBSA>().First(b => b.TempID == bsa_id);
|
||||
|
||||
using (var a = BSADispatch.OpenRead(Path.Combine(MO2Folder, bsa.To)))
|
||||
using (var a = await BSADispatch.OpenRead(Path.Combine(MO2Folder, bsa.To)))
|
||||
{
|
||||
var find = Path.Combine(to.Split('\\').Skip(2).ToArray());
|
||||
var file = a.Files.First(e => e.Path.Replace('/', '\\') == find);
|
||||
using (var ms = new MemoryStream())
|
||||
{
|
||||
file.CopyDataTo(ms);
|
||||
await file.CopyDataToAsync(ms);
|
||||
return ms.ToArray();
|
||||
}
|
||||
}
|
||||
|
||||
@ -400,7 +400,21 @@ namespace Wabbajack.Lib
|
||||
void CopyFile(string from, string to, bool use_move)
|
||||
{
|
||||
if (File.Exists(to))
|
||||
{
|
||||
var fi = new FileInfo(to);
|
||||
if (fi.IsReadOnly)
|
||||
fi.IsReadOnly = false;
|
||||
File.Delete(to);
|
||||
}
|
||||
|
||||
if (File.Exists(from))
|
||||
{
|
||||
var fi = new FileInfo(from);
|
||||
if (fi.IsReadOnly)
|
||||
fi.IsReadOnly = false;
|
||||
}
|
||||
|
||||
|
||||
if (use_move)
|
||||
File.Move(from, to);
|
||||
else
|
||||
|
||||
@ -24,9 +24,6 @@ namespace Wabbajack.Lib.NexusApi
|
||||
{
|
||||
private static readonly string API_KEY_CACHE_FILE = "nexus.key_cache";
|
||||
|
||||
private static readonly uint CACHED_VERSION_NUMBER = 1;
|
||||
|
||||
|
||||
private readonly HttpClient _httpClient;
|
||||
|
||||
|
||||
|
||||
252
Wabbajack.Test/CSP/CSPTests.cs
Normal file
252
Wabbajack.Test/CSP/CSPTests.cs
Normal file
@ -0,0 +1,252 @@
|
||||
using System;
|
||||
using System.Linq;
|
||||
using System.Reactive.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Alphaleonis.Win32.Filesystem;
|
||||
using Microsoft.VisualStudio.TestTools.UnitTesting;
|
||||
using Wabbajack.Common.CSP;
|
||||
|
||||
namespace Wabbajack.Test.CSP
|
||||
{
|
||||
[TestClass]
|
||||
public class CSPTests
|
||||
{
|
||||
|
||||
public TestContext TestContext { get; set; }
|
||||
|
||||
public void Log(string msg)
|
||||
{
|
||||
TestContext.WriteLine(msg);
|
||||
}
|
||||
|
||||
[TestInitialize]
|
||||
public void Startup()
|
||||
{
|
||||
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Test that we can put a value onto a channel without a buffer, and that the put is released once the
|
||||
/// take finalizes
|
||||
/// </summary>
|
||||
/// <returns></returns>
|
||||
[TestMethod]
|
||||
public async Task TestTakePutBlocking()
|
||||
{
|
||||
var channel = Channel.Create<int>();
|
||||
var ptask = channel.Put(1);
|
||||
var (open, val) = await channel.Take();
|
||||
|
||||
Assert.AreEqual(1, val);
|
||||
Assert.IsTrue(open);
|
||||
Assert.IsTrue(await ptask);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// If we create a channel with a fixed buffer size, we can enqueue that number of items without blocking
|
||||
/// We can then take those items later on.
|
||||
/// </summary>
|
||||
/// <returns></returns>
|
||||
[TestMethod]
|
||||
public async Task TestTakePutBuffered()
|
||||
{
|
||||
var channel = Channel.Create<int>(10);
|
||||
foreach (var itm in Enumerable.Range(0, 10))
|
||||
await channel.Put(itm);
|
||||
|
||||
foreach (var itm in Enumerable.Range(0, 10))
|
||||
{
|
||||
var (is_open, val) = await channel.Take();
|
||||
Assert.AreEqual(itm, val);
|
||||
Assert.IsTrue(is_open);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// We can convert a IEnumerable into a channel by inlining the enumerable into the channel's buffer.
|
||||
/// </summary>
|
||||
/// <returns></returns>
|
||||
[TestMethod]
|
||||
public async Task TestToChannel()
|
||||
{
|
||||
var channel = Enumerable.Range(0, 10).ToChannel();
|
||||
|
||||
foreach (var itm in Enumerable.Range(0, 10))
|
||||
{
|
||||
var (is_open, val) = await channel.Take();
|
||||
Assert.AreEqual(itm, val);
|
||||
Assert.IsTrue(is_open);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// TakeAll will continue to take from a channel as long as the channel is open. Once the channel closes
|
||||
/// TakeAll returns a list of the items taken.
|
||||
/// </summary>
|
||||
/// <returns></returns>
|
||||
[TestMethod]
|
||||
public async Task TestTakeAll()
|
||||
{
|
||||
var results = await Enumerable.Range(0, 10).ToChannel().TakeAll();
|
||||
|
||||
CollectionAssert.AreEqual(Enumerable.Range(0, 10).ToList(), results);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// We can add Rx transforms as transforms inside a channel. This allows for cheap conversion and calcuation
|
||||
/// to be performed in a channel without incuring the dispatch overhead of swapping values between threads.
|
||||
/// These calculations happen inside the channel's lock, however, so be sure to keep these operations relatively
|
||||
/// cheap.
|
||||
/// </summary>
|
||||
/// <returns></returns>
|
||||
[TestMethod]
|
||||
public async Task RxTransformInChannel()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, o => o.Select(v => v + 1));
|
||||
var finished = Enumerable.Range(0, 10).OntoChannel(chan);
|
||||
|
||||
foreach (var itm in Enumerable.Range(0, 10))
|
||||
{
|
||||
var (is_open, val) = await chan.Take();
|
||||
Assert.AreEqual(itm + 1, val);
|
||||
Assert.IsTrue(is_open);
|
||||
}
|
||||
await finished;
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task UnorderedPipeline()
|
||||
{
|
||||
var o = Channel.Create<string>(3);
|
||||
var finished = Enumerable.Range(0, 3)
|
||||
.ToChannel()
|
||||
.UnorderedPipeline(1, o, obs => obs.Select(itm => itm.ToString()));
|
||||
|
||||
var results = (await o.TakeAll()).OrderBy(e => e).ToList();
|
||||
var expected = Enumerable.Range(0, 3).Select(i => i.ToString()).OrderBy(e => e).ToList();
|
||||
CollectionAssert.AreEqual(expected, results);
|
||||
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task UnorderedPipelineWithParallelism()
|
||||
{
|
||||
// Do it a hundred times to try and catch rare deadlocks
|
||||
var o = Channel.Create<string>(3);
|
||||
var finished = Enumerable.Range(0, 1024)
|
||||
.ToChannel()
|
||||
.UnorderedPipeline(4, o, obs => obs.Select(itm => itm.ToString()));
|
||||
|
||||
var results = (await o.TakeAll()).OrderBy(e => e).ToList();
|
||||
var expected = Enumerable.Range(0, 1024).Select(i => i.ToString()).OrderBy(e => e).ToList();
|
||||
CollectionAssert.AreEqual(expected, results);
|
||||
await finished;
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task UnorderedTaskPipeline()
|
||||
{
|
||||
// Do it a hundred times to try and catch rare deadlocks
|
||||
var o = Channel.Create<int>(3);
|
||||
var finished = Enumerable.Range(0, 1024)
|
||||
.ToChannel()
|
||||
.UnorderedPipeline(4, o, async v =>
|
||||
{
|
||||
await Task.Delay(1);
|
||||
return v;
|
||||
});
|
||||
|
||||
var results = (await o.TakeAll()).OrderBy(e => e).ToList();
|
||||
var expected = Enumerable.Range(0, 1024).ToList();
|
||||
CollectionAssert.AreEqual(expected, results);
|
||||
await finished;
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task UnorderedThreadPipeline()
|
||||
{
|
||||
// Do it a hundred times to try and catch rare deadlocks
|
||||
var o = Channel.Create<int>(3);
|
||||
var finished = Enumerable.Range(0, 1024)
|
||||
.ToChannel()
|
||||
.UnorderedThreadedPipeline(4, o, v =>
|
||||
{
|
||||
Thread.Sleep(1);
|
||||
return v;
|
||||
});
|
||||
|
||||
var results = (await o.TakeAll()).OrderBy(e => e).ToList();
|
||||
var expected = Enumerable.Range(0, 1024).ToList();
|
||||
CollectionAssert.AreEqual(expected, results);
|
||||
await finished;
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task ChannelStressTest()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
var putter = Task.Run(async () =>
|
||||
{
|
||||
for (var i = 0; i < 1000; i++)
|
||||
{
|
||||
var result = await chan.Put(i);
|
||||
}
|
||||
});
|
||||
|
||||
var taker = Task.Run(async () =>
|
||||
{
|
||||
try
|
||||
{
|
||||
for (var i = 0; i < 1000; i++)
|
||||
{
|
||||
var (is_open, val) = await chan.Take();
|
||||
Assert.AreEqual(i, val);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
chan.Close();
|
||||
}
|
||||
});
|
||||
|
||||
await putter;
|
||||
await taker;
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task ChannelStressWithBuffer()
|
||||
{
|
||||
var chan = Channel.Create<int>(1);
|
||||
|
||||
var putter = Task.Run(async () =>
|
||||
{
|
||||
for (var i = 0; i < 1000; i++)
|
||||
{
|
||||
await chan.Put(i);
|
||||
}
|
||||
});
|
||||
|
||||
var taker = Task.Run(async () =>
|
||||
{
|
||||
try
|
||||
{
|
||||
for (var i = 0; i < 1000; i++)
|
||||
{
|
||||
var (is_open, val) = await chan.Take();
|
||||
Assert.AreEqual(i, val);
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
chan.Close();
|
||||
}
|
||||
});
|
||||
|
||||
await putter;
|
||||
await taker;
|
||||
}
|
||||
}
|
||||
}
|
||||
239
Wabbajack.Test/CSP/ChannelTests.cs
Normal file
239
Wabbajack.Test/CSP/ChannelTests.cs
Normal file
@ -0,0 +1,239 @@
|
||||
using System.Linq;
|
||||
using System.Reactive.Linq;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.VisualStudio.TestTools.UnitTesting;
|
||||
using Wabbajack.Common.CSP;
|
||||
|
||||
namespace Wabbajack.Test.CSP
|
||||
{
|
||||
[TestClass]
|
||||
public class ChannelTests
|
||||
{
|
||||
[TestMethod]
|
||||
public async Task PutThenTakeNoBuffer()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
var putter = chan.Put(42);
|
||||
var taker = chan.Take();
|
||||
|
||||
Assert.IsTrue(await putter);
|
||||
Assert.AreEqual((true, 42), await taker);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TakeThenPushNoBuffer()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
var taker = chan.Take();
|
||||
var putter = chan.Put(42);
|
||||
|
||||
Assert.IsTrue(await putter);
|
||||
Assert.AreEqual((true, 42), await taker);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TakeFromBufferAfterPut()
|
||||
{
|
||||
var chan = Channel.Create<int>(1);
|
||||
|
||||
var putter = chan.Put(42);
|
||||
var taker = chan.Take();
|
||||
|
||||
Assert.IsTrue(await putter);
|
||||
Assert.AreEqual((true, 42), await taker);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TakeFromBufferBeforePut()
|
||||
{
|
||||
var chan = Channel.Create<int>(1);
|
||||
|
||||
var taker = chan.Take();
|
||||
var putter = chan.Put(42);
|
||||
|
||||
Assert.IsTrue(await putter);
|
||||
Assert.AreEqual((true, 42), await taker);
|
||||
}
|
||||
|
||||
|
||||
[TestMethod]
|
||||
public async Task TakesAreReleasedAfterClose()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
var taker = chan.Take();
|
||||
chan.Close();
|
||||
|
||||
Assert.AreEqual((false, 0), await taker);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task ExpandingTransformsReleaseMultipleTakes()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.SelectMany(len => Enumerable.Range(0, len)));
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
await chan.Put(2);
|
||||
|
||||
Assert.AreEqual((true, 0), await take1);
|
||||
Assert.AreEqual((true, 1), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanCloseChannel()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Take(1));
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
|
||||
await chan.Put(1);
|
||||
await chan.Put(2);
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((true, 1), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanCloseDuringExpand()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.SelectMany(len => Enumerable.Range(1, len)).Take(1));
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
|
||||
await chan.Put(2);
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((true, 1), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanFilterTakeFirst()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Where(x => x == 2));
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
|
||||
await chan.Put(1);
|
||||
await chan.Put(2);
|
||||
chan.Close();
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((true, 2), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanReturnNothingTakeFirst()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Take(0));
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
|
||||
await chan.Put(1);
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((false, 0), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanFilterTakeAfter()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Where(x => x == 2));
|
||||
|
||||
|
||||
await chan.Put(1);
|
||||
await chan.Put(2);
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
chan.Close();
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((true, 2), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task TransformsCanReturnNothingTakeAfter()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Take(0));
|
||||
|
||||
await chan.Put(1);
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
|
||||
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
|
||||
Assert.AreEqual((false, 0), await take1);
|
||||
Assert.AreEqual((false, 0), await take2);
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public void TooManyTakesCausesException()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
Assert.ThrowsException<ManyToManyChannel<int, int>.TooManyHanldersException>(() =>
|
||||
{
|
||||
for (var x = 0; x < ManyToManyChannel<int, int>.MAX_QUEUE_SIZE + 1; x++)
|
||||
chan.Take();
|
||||
});
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public void TooManyPutsCausesException()
|
||||
{
|
||||
var chan = Channel.Create<int>();
|
||||
|
||||
Assert.ThrowsException<ManyToManyChannel<int, int>.TooManyHanldersException>(() =>
|
||||
{
|
||||
for (var x = 0; x < ManyToManyChannel<int, int>.MAX_QUEUE_SIZE + 1; x++)
|
||||
chan.Put(x);
|
||||
});
|
||||
}
|
||||
|
||||
[TestMethod]
|
||||
public async Task BlockingPutsGoThroughTransform()
|
||||
{
|
||||
var chan = Channel.Create<int, int>(1, i => i.Take(2));
|
||||
|
||||
var put1 = chan.Put(1);
|
||||
var put2 = chan.Put(2);
|
||||
var put3 = chan.Put(3);
|
||||
var put4 = chan.Put(4);
|
||||
|
||||
var take1 = chan.Take();
|
||||
var take2 = chan.Take();
|
||||
var take3 = chan.Take();
|
||||
|
||||
|
||||
|
||||
Assert.AreEqual((true, 1), await take1);
|
||||
Assert.AreEqual((true, 2), await take2);
|
||||
Assert.AreEqual((false, 0), await take3);
|
||||
|
||||
Assert.IsTrue(await put1);
|
||||
Assert.IsTrue(await put2);
|
||||
Assert.IsFalse(await put3);
|
||||
Assert.IsFalse(await put4);
|
||||
Assert.IsTrue(chan.IsClosed);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -93,9 +93,9 @@ namespace Wabbajack.Test
|
||||
File.Copy(src, Path.Combine(utils.DownloadsFolder, filename));
|
||||
|
||||
if (mod_name == null)
|
||||
FileExtractor.ExtractAll(src, utils.MO2Folder);
|
||||
FileExtractor.ExtractAll(src, utils.MO2Folder).Wait();
|
||||
else
|
||||
FileExtractor.ExtractAll(src, Path.Combine(utils.ModsFolder, mod_name));
|
||||
FileExtractor.ExtractAll(src, Path.Combine(utils.ModsFolder, mod_name)).Wait();
|
||||
|
||||
}
|
||||
|
||||
@ -115,7 +115,7 @@ namespace Wabbajack.Test
|
||||
$"fileID={file.file_id}"
|
||||
});
|
||||
|
||||
if (!File.Exists(file.file_name))
|
||||
if (!File.Exists(src))
|
||||
{
|
||||
|
||||
var state = DownloadDispatcher.ResolveArchive(ini.LoadIniString());
|
||||
@ -130,7 +130,7 @@ namespace Wabbajack.Test
|
||||
var dest = Path.Combine(utils.DownloadsFolder, file.file_name);
|
||||
File.Copy(src, dest);
|
||||
|
||||
FileExtractor.ExtractAll(src, Path.Combine(utils.ModsFolder, mod_name));
|
||||
FileExtractor.ExtractAll(src, Path.Combine(utils.ModsFolder, mod_name)).Wait();
|
||||
|
||||
File.WriteAllText(dest + ".meta", ini);
|
||||
}
|
||||
|
||||
19
Wabbajack.Test/Wabbajack.Common.Tests/ChannelStreamsTests.cs
Normal file
19
Wabbajack.Test/Wabbajack.Common.Tests/ChannelStreamsTests.cs
Normal file
@ -0,0 +1,19 @@
|
||||
using System;
|
||||
using System.Linq;
|
||||
using Microsoft.VisualStudio.TestTools.UnitTesting;
|
||||
using Wabbajack.Common;
|
||||
|
||||
namespace Wabbajack.Test.Wabbajack.Common.Tests
|
||||
{
|
||||
[TestClass]
|
||||
public class ChannelStreamsTests
|
||||
{
|
||||
[TestMethod]
|
||||
public void ToAndFromChannel()
|
||||
{
|
||||
var src = Enumerable.Range(0, 10).ToList();
|
||||
var result = src.AsChannel().ToIEnumerable();
|
||||
Assert.AreEqual(src, result);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -93,6 +93,8 @@
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Compile Include="ACompilerTest.cs" />
|
||||
<Compile Include="CSP\ChannelTests.cs" />
|
||||
<Compile Include="CSP\CSPTests.cs" />
|
||||
<Compile Include="DownloaderTests.cs" />
|
||||
<Compile Include="EndToEndTests.cs" />
|
||||
<Compile Include="Extensions.cs" />
|
||||
@ -113,6 +115,10 @@
|
||||
<Project>{5128b489-bc28-4f66-9f0b-b4565af36cbc}</Project>
|
||||
<Name>VirtualFileSystem</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Common.CSP\Wabbajack.Common.CSP.csproj">
|
||||
<Project>{9e69bc98-1512-4977-b683-6e7e5292c0b8}</Project>
|
||||
<Name>Wabbajack.Common.CSP</Name>
|
||||
</ProjectReference>
|
||||
<ProjectReference Include="..\Wabbajack.Common\Wabbajack.Common.csproj">
|
||||
<Project>{b3f3fb6e-b9eb-4f49-9875-d78578bc7ae5}</Project>
|
||||
<Name>Wabbajack.Common</Name>
|
||||
@ -146,6 +152,7 @@
|
||||
<Version>4.2.0</Version>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
<ItemGroup />
|
||||
<Import Project="$(VSToolsPath)\TeamTest\Microsoft.TestTools.targets" Condition="Exists('$(VSToolsPath)\TeamTest\Microsoft.TestTools.targets')" />
|
||||
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
|
||||
</Project>
|
||||
@ -9,8 +9,6 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wabbajack", "Wabbajack\Wabb
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Compression.BSA", "Compression.BSA\Compression.BSA.csproj", "{FF5D892F-8FF4-44FC-8F7F-CD58F307AD1B}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Compression.BSA.Test", "Compression.BSA.Test\Compression.BSA.Test.csproj", "{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}"
|
||||
EndProject
|
||||
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{4EDEF6CC-2F5C-439B-BEAF-9D03895099F1}"
|
||||
ProjectSection(SolutionItems) = preProject
|
||||
CHANGELOG.md = CHANGELOG.md
|
||||
@ -30,6 +28,10 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wabbajack.Lib", "Wabbajack.
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wabbajack.Test.ListValidation", "Wabbajack.Test.ListValidation\Wabbajack.Test.ListValidation.csproj", "{BA013D05-1D70-452F-BB8F-272B31E6C74E}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Compression.BSA.Test", "Compression.BSA.Test\Compression.BSA.Test.csproj", "{9C004392-571A-4D28-A9F6-0E25115E6727}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Wabbajack.Common.CSP", "Wabbajack.Common.CSP\Wabbajack.Common.CSP.csproj", "{9E69BC98-1512-4977-B683-6E7E5292C0B8}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
Debug (no commandargs)|Any CPU = Debug (no commandargs)|Any CPU
|
||||
@ -97,24 +99,6 @@ Global
|
||||
{FF5D892F-8FF4-44FC-8F7F-CD58F307AD1B}.Release|x64.Build.0 = Release|x64
|
||||
{FF5D892F-8FF4-44FC-8F7F-CD58F307AD1B}.Release|x86.ActiveCfg = Release|x86
|
||||
{FF5D892F-8FF4-44FC-8F7F-CD58F307AD1B}.Release|x86.Build.0 = Release|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|Any CPU.Build.0 = Debug|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|x64.ActiveCfg = Debug|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|x64.Build.0 = Debug|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|x86.ActiveCfg = Debug|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug (no commandargs)|x86.Build.0 = Debug|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|x64.Build.0 = Debug|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|x86.ActiveCfg = Debug|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Debug|x86.Build.0 = Debug|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|x64.ActiveCfg = Release|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|x64.Build.0 = Release|x64
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|x86.ActiveCfg = Release|x86
|
||||
{BA2CFEA1-072B-42D6-822A-8C6D0E3AE5D9}.Release|x86.Build.0 = Release|x86
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug (no commandargs)|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug (no commandargs)|Any CPU.Build.0 = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug (no commandargs)|x64.ActiveCfg = Debug|x64
|
||||
@ -123,8 +107,8 @@ Global
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug (no commandargs)|x86.Build.0 = Debug|x86
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x64.Build.0 = Debug|x64
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x86.ActiveCfg = Debug|x86
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Debug|x86.Build.0 = Debug|x86
|
||||
{5128B489-BC28-4F66-9F0B-B4565AF36CBC}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
@ -141,8 +125,8 @@ Global
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug (no commandargs)|x86.Build.0 = Debug|x86
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x64.Build.0 = Debug|x64
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x86.ActiveCfg = Debug|x86
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Debug|x86.Build.0 = Debug|x86
|
||||
{A2913DFE-18FF-468B-A6C1-55F7C0CC0CE8}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
@ -159,8 +143,8 @@ Global
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug (no commandargs)|x86.Build.0 = Debug|x86
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x64.Build.0 = Debug|x64
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x86.ActiveCfg = Debug|x86
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Debug|x86.Build.0 = Debug|x86
|
||||
{A47FFF32-782B-4D9F-8704-C98FB32FA8CC}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
@ -177,8 +161,8 @@ Global
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug (no commandargs)|x86.Build.0 = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x64.Build.0 = Debug|x64
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{0A820830-A298-497D-85E0-E9A89EFEF5FE}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
@ -195,8 +179,8 @@ Global
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug (no commandargs)|x86.Build.0 = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x64.Build.0 = Debug|x64
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
@ -205,6 +189,42 @@ Global
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Release|x64.Build.0 = Release|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Release|x86.ActiveCfg = Release|Any CPU
|
||||
{BA013D05-1D70-452F-BB8F-272B31E6C74E}.Release|x86.Build.0 = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|Any CPU.Build.0 = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|x64.ActiveCfg = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|x64.Build.0 = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|x86.ActiveCfg = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug (no commandargs)|x86.Build.0 = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|x64.Build.0 = Debug|x64
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|x64.ActiveCfg = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|x64.Build.0 = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|x86.ActiveCfg = Release|Any CPU
|
||||
{9C004392-571A-4D28-A9F6-0E25115E6727}.Release|x86.Build.0 = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|Any CPU.Build.0 = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|x64.ActiveCfg = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|x64.Build.0 = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|x86.ActiveCfg = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug (no commandargs)|x86.Build.0 = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|x64.ActiveCfg = Debug|x64
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|x64.Build.0 = Debug|x64
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|x64.ActiveCfg = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|x64.Build.0 = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|x86.ActiveCfg = Release|Any CPU
|
||||
{9E69BC98-1512-4977-B683-6E7E5292C0B8}.Release|x86.Build.0 = Release|Any CPU
|
||||
EndGlobalSection
|
||||
GlobalSection(SolutionProperties) = preSolution
|
||||
HideSolutionNode = FALSE
|
||||
|
||||
@ -54,8 +54,9 @@ namespace Wabbajack
|
||||
|
||||
// Set up logging
|
||||
Utils.LogMessages
|
||||
.ObserveOn(RxApp.TaskpoolScheduler)
|
||||
.ToObservableChangeSet()
|
||||
.Buffer(TimeSpan.FromMilliseconds(250))
|
||||
.Buffer(TimeSpan.FromMilliseconds(250), RxApp.TaskpoolScheduler)
|
||||
.Where(l => l.Count > 0)
|
||||
.FlattenBufferResult()
|
||||
.Top(5000)
|
||||
@ -89,7 +90,7 @@ namespace Wabbajack
|
||||
WorkQueue.Status
|
||||
.ObserveOn(RxApp.TaskpoolScheduler)
|
||||
.ToObservableChangeSet(x => x.ID)
|
||||
.Batch(TimeSpan.FromMilliseconds(250))
|
||||
.Batch(TimeSpan.FromMilliseconds(250), RxApp.TaskpoolScheduler)
|
||||
.EnsureUniqueChanges()
|
||||
.ObserveOn(RxApp.MainThreadScheduler)
|
||||
.Sort(SortExpressionComparer<CPUStatus>.Ascending(s => s.ID), SortOptimisations.ComparesImmutableValuesOnly)
|
||||
|
||||
@ -388,7 +388,7 @@
|
||||
<Version>4.1.0</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="DynamicData">
|
||||
<Version>6.13.20</Version>
|
||||
<Version>6.13.21</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="Fody">
|
||||
<Version>6.0.4</Version>
|
||||
@ -425,7 +425,7 @@
|
||||
<Version>2.4.0</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="ReactiveUI.Events.WPF">
|
||||
<Version>10.4.1</Version>
|
||||
<Version>10.5.7</Version>
|
||||
</PackageReference>
|
||||
<PackageReference Include="ReactiveUI.Fody">
|
||||
<Version>10.5.7</Version>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user