File tree Expand file tree Collapse file tree 3 files changed +18
-9
lines changed
Expand file tree Collapse file tree 3 files changed +18
-9
lines changed Original file line number Diff line number Diff line change 1515 - name : Setup .NET
1616 uses : actions/setup-dotnet@v1
1717 with :
18- dotnet-version : 6 .0.x
18+ dotnet-version : 8 .0.x
1919 - run : dotnet restore
2020 - run : dotnet test UnitTests/UnitTests.csproj
2121
2727 - name : Setup .NET
2828 uses : actions/setup-dotnet@v1
2929 with :
30- dotnet-version : 6 .0.x
30+ dotnet-version : 8 .0.x
3131 - name : Build
3232 run : dotnet build --configuration Release
3333 - name : Publish
4545 env :
4646 GITHUB_TOKEN : ${{ secrets.GITHUB_TOKEN }}
4747 with :
48- tag_name : v2.0.6
49- release_name : v2.0.6
48+ tag_name : v2.0.7
49+ release_name : v2.0.7
5050 draft : false
5151 prerelease : false
5252
Original file line number Diff line number Diff line change 55 <ImplicitUsings >enable</ImplicitUsings >
66 <Nullable >enable</Nullable >
77 <PackageId >Nick.RobotsParser</PackageId >
8- <Version >2.0.6 </Version >
8+ <Version >2.0.7 </Version >
99 <Authors >Nicholas Bergesen</Authors >
1010 <Company >Nicholas Bergesen</Company >
1111 <PackageReleaseNotes >
Original file line number Diff line number Diff line change 33** Nuget Page** : https://www.nuget.org/packages/Nick.RobotsParser/
44
55## Install options
6- - Install-Package Nick.RobotsParser -Version 2.0.6
7- - dotnet add package Nick.RobotsParser --version 2.0.6
6+ - Install-Package Nick.RobotsParser -Version 2.0.7
7+ - dotnet add package Nick.RobotsParser --version 2.0.7
88
99## Example Snippets
1010### Load robots.txt
1111```
12- var robots = new Robots(websiteUri: "https://websiteurl.com", userAgent: "my custom user agent");
13- await robots.Load();
12+ public static async Task<string> GetHttpResponseBodyAsync(string url)
13+ {
14+ using var httpClient = new HttpClient();
15+ HttpResponseMessage response = await httpClient.GetAsync(url);
16+ response.EnsureSuccessStatusCode();
17+ string responseBody = await response.Content.ReadAsStringAsync();
18+ return responseBody;
19+ }
20+
21+ var robots = new Robots(downloadFunc: GetHttpResponseBodyAsync);
22+ await robots.LoadRobotsFromUrl("https://www.google.com/robots.txt");
1423```
1524
1625### Get robots info
You can’t perform that action at this time.
0 commit comments